DEEP LEARNING IN HUMAN POSE GENERATION AND ITS APPLICATION

Abstract

This thesis targets the challenge of enhancing the performance of human pose estimation in the face of data limitation, placing an emphasis on generating diverse and plausible pose training data. Human pose estimation, which estimates 3D poses from images or videos, has a wide array of applications including action recognition, human tracking, and mixed reality. Nevertheless, the effectiveness of this estimation in real-world scenarios is often compromised due to the limited diversity of indoor training data primarily available. To address this, we firstly explore data augmentation based on existing 3D training data to generate diverse and plausible augmentation data. We then tackle a more challenging scenario of training a 3D pose estimator using diverse and plausible 3D data generated solely from 2D data in a self-supervised manner. Lastly, we explore the potential of diverse and plausible data generation from other modalities, such as text, music, or both.