3D Human Pose Estimation in the Wild by Adversarial Learning

Abstract

Recently, remarkable advances have been achieved in 3D human pose estimation from monocular images because of the powerful Deep Convolutional Neural Networks (DCNNs). Despite their success on large-scale datasets collected in the constrained lab environment, it is difficult to obtain the 3D pose annotations for in-the-wild images. Therefore, 3D human pose estimation in the wild is still a challenge. In this paper, we propose an adversarial learning framework, which distills the 3D human pose structures learned from the fully annotated dataset to in-the-wild images with only 2D pose annotations. Instead of defining hard-coded rules to constrain the pose estimation results, we design a novel multi-source discriminator to distinguish the predicted 3D poses from the ground-truth, which helps to enforce the pose estimator to generate anthropometrically valid poses even with images in the wild. We also observe that a carefully designed information source for the discriminator is essential to boost the performance. Thus, we design a geometric descriptor, which computes the pairwise relative locations and distances between body joints, as a new information source for the discriminator. The efficacy of our adversarial learning framework with the new geometric descriptor has been demonstrated through extensive experiments on widely used public benchmarks. Our approach significantly improves the performance compared with previous state-of-the-art approaches.

arXiv commentCVPR 2018

References in corpus (22)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• DeepPose: Human Pose Estimation via Deep Neural Networks
• Unsupervised Domain Adaptation by Backpropagation
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera
• Generating Videos with Scene Dynamics
• Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image
• Stacked Hourglass Networks for Human Pose Estimation
• Convolutional Pose Machines
• Learning to Generate Long-term Future via Hierarchical Prediction
• Multi-Context Attention for Human Pose Estimation
• Recurrent Topic-Transition GAN for Visual Paragraph Generation
• Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach
• Learning Feature Pyramids for Human Pose Estimation
• Adversarial PoseNet: A Structure-aware Convolutional Network for Human Pose Estimation
• Structured Prediction of 3D Human Pose with Deep Neural Networks
• Stacked Generative Adversarial Networks
• Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
• Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision
• 3D Human Pose Estimation from a Single Image via Distance Matrix Regression
• Adversarial Learning of Structure-Aware Fully Convolutional Networks for Landmark Localization

Cited by in corpus (12)

• Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimation
• 3D Human Pose Estimation with Relational Networks
• Monocular Total Capture: Posing Face, Body, and Hands in the Wild
• DeepHuman: 3D Human Reconstruction from a Single Image
• Inference Stage Optimization for Cross-scenario 3D Human Pose Estimation
• An Adversarial Learning Approach to Medical Image Synthesis for Lesion Detection
• PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation
• A Graph Attention Spatio-temporal Convolutional Network for 3D Human Pose Estimation in Video
• FBI-Pose: Towards Bridging the Gap between 2D Images and 3D Human Poses using Forward-or-Backward Information
• View Invariant 3D Human Pose Estimation
• Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB
• Adversarial 3D Human Pose Estimation via Multimodal Depth Supervision