3D Human Pose Estimation Based on 2D-3D Consistency with Synchronized Adversarial Training

Abstract

3D human pose estimation from a single image is still a challenging problem despite the large amount of work that has been performed in this field. Generally, most methods directly use neural networks and ignore certain constraints (e.g., reprojection constraints, joint angle, and bone length constraints). While a few methods consider these constraints but train the network separately, they cannot effectively solve the depth ambiguity problem. In this paper, we propose a GAN-based model for 3D human pose estimation, in which a reprojection network is employed to learn the mapping of the distribution from 3D poses to 2D poses, and a discriminator is employed for 2D-3D consistency discrimination. We adopt a novel strategy to synchronously train the generator, the reprojection network and the discriminator. Furthermore, inspired by the typical kinematic chain space (KCS) matrix, we introduce a weighted KCS matrix and take it as one of the discriminator's inputs to impose joint angle and bone length constraints. The experimental results on Human3.6M show that our method significantly outperforms state-of-the-art methods in most cases.

References in corpus (8)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• DeepPose: Human Pose Estimation via Deep Neural Networks
• Empirical Evaluation of Rectified Activations in Convolutional Network
• VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera
• Human pose estimation via Convolutional Part Heatmap Regression
• Sparse Representation for 3D Shape Estimation: A Convex Relaxation Approach
• OriNet: A Fully Convolutional Network for 3D Human Pose Estimation
• Learning Pose Grammar to Encode Human Body Configuration for 3D Pose Estimation