NON-PARAMETRIC 3D HAND SHAPE RECONSTRUCTION FROM MONOCULAR IMAGE

Abstract

This study focuses on 3D hand shape estimation from RGB input, crucial for human-computer interaction and AR/VR. Deep learning advancements have enhanced this field, often relying on parametric hand models like MANO. However, these models diverge from real hand surfaces and demand extensive annotations. To address this, non-parametric methods are explored, incorporating point cloud and UV map representations. A novel UV-map-based pipeline improves accuracy while considering realistic hand-object interactions. Template-based methods excel but require predefined object models. We propose a category-level 3D hand-object reconstruction framework, incorporating shape priors for unseen objects. Previous approaches fall into parametric or non-parametric categories, each with limitations. Our thesis integrates both for accurate reconstruction and introduces a VAE correction module. Additionally, a weakly-supervised pipeline leverages 3D joint labels, advancing 3D hand shape estimation further.