Human pose completion in partial body camera shots

ABSTRACT

Many actual images contain partial body camera shots, in which a significant part of the body is not visible. This issue is especially prevalent in film images, where less than 10% are full-body shots. Most 2D human pose estimation methods return incomplete poses when applied to partial body images. This lack of completeness becomes a problem in some situations, for example, when the 2D pose is converted to a 3D pose by a two-stage 3D human pose estimation method since most of these methods require a complete pose to work. This article proposes a new technique, called CompletePose, consisting of completing the missing keypoints when 2D human pose estimation methods are applied to images with partial body camera shots. A Conditional Generative Adversarial Network is used to obtain a complete and plausible pose, realistic enough to predict a 3D pose with a two-stage 3D human pose estimation method. A complete empirical validation has been carried out with the Human3.6 M dataset and a new dataset, called CHARADE, specially built and made public for reproducibility and benchmarking for this research. Quantitative evaluation employing the Fréchet distance shows that the approach manages to approximate the actual data distribution. The qualitative evaluation shows that the completed poses enable obtaining plausible 3D poses from images previously intractable.

KEYWORDS:

• Human pose estimation
• human pose completion
• movies
• generative adversarial networks
• deep learning

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. https://wonderdynamics.com

2. https://plask.ai/

3. Tous (2023).

4. When applying the method over OpenPose results, the keypoints for eyes and ears (not included in Human3.6 M) are directly restored from the OpenPose output.

Additional information

Funding

This work is partially supported by the Spanish Ministry of Science and Innovation under contracts PID2019-107255GB, PLEC2021-007850 and RTI2018-095232-B-C21, and by the SGR programme of the Catalan Government under contracts 2021-SGR-00478, 2017-SGR-1414 and 2017-SGR-1742.