Adversarial Learning of Structure-Aware Fully Convolutional Networks for Landmark Localization

Abstract

Landmark/pose estimation in single monocular images have received much effort in computer vision due to its important applications. It remains a challenging task when input images severe occlusions caused by, e.g., adverse camera views. Under such circumstances, biologically implausible pose predictions may be produced. In contrast, human vision is able to predict poses by exploiting geometric constraints of landmark point inter-connectivity. To address the problem, by incorporating priors about the structure of pose components, we propose a novel structure-aware fully convolutional network to implicitly take such priors into account during training of the deep network. Explicit learning of such constraints is typically challenging. Instead, inspired by how human identifies implausible poses, we design discriminators to distinguish the real poses from the fake ones (such as biologically implausible ones). If the pose generator G generates results that the discriminator fails to distinguish from real ones, the network successfully learns the priors. Training of the network follows the strategy of conditional Generative Adversarial Networks (GANs). The effectiveness of the proposed network is evaluated on three pose-related tasks: 2D single human pose estimation, 2D facial landmark estimation and 3D single human pose estimation. The proposed approach significantly outperforms the state-of-the-art methods and almost always generates plausible pose predictions, demonstrating the usefulness of implicit learning of structures using GANs.

arXiv comment18 pages. Extended version of arXiv:1705.00389. Accepted to IEEE Trans. Pattern Analysis and Machine Intelligence

References in corpus (9)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• DeepPose: Human Pose Estimation via Deep Neural Networks
• Generative Adversarial Text to Image Synthesis
• Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
• Energy-based Generative Adversarial Network
• Human pose estimation via Convolutional Part Heatmap Regression
• Sparse Representation for 3D Shape Estimation: A Convex Relaxation Approach

Cited by in corpus (6)

• Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning
• Single-shot 3D multi-person pose estimation in complex images
• FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors
• 3D Human Pose Estimation in the Wild by Adversarial Learning
• Multi-scale Aggregation R-CNN for 2D Multi-person Pose Estimation
• Towards Highly Accurate and Stable Face Alignment for High-Resolution Videos