3D Human Pose Estimation from a Single Image via Distance Matrix Regression

Abstract

This paper addresses the problem of 3D human pose estimation from a single image. We follow a standard two-step pipeline by first detecting the 2D position of the $N$ body joints, and then using these observations to infer 3D pose. For the first step, we use a recent CNN-based detector. For the second step, most existing approaches perform 2$N$-to-3$N$ regression of the Cartesian joint coordinates. We show that more precise pose estimates can be obtained by representing both the 2D and 3D human poses using $N\times N$ distance matrices, and formulating the problem as a 2D-to-3D distance matrix regression. For learning such a regressor we leverage on simple Neural Network architectures, which by construction, enforce positivity and symmetry of the predicted matrices. The approach has also the advantage to naturally handle missing observations and allowing to hypothesize the position of non-observed joints. Quantitative results on Humaneva and Human3.6M datasets demonstrate consistent performance gains over state-of-the-art. Qualitative evaluation on the images in-the-wild of the LSP dataset, using the regressor learned on Human3.6M, reveals very promising generalization results.

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Fully Convolutional Networks for Semantic Segmentation
• MatConvNet - Convolutional Neural Networks for MATLAB
• Learning Deconvolution Network for Semantic Segmentation
• FlowNet: Learning Optical Flow with Convolutional Networks
• Convolutional Pose Machines
• MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild
• Sparse Representation for 3D Shape Estimation: A Convex Relaxation Approach
• Structured Prediction of 3D Human Pose with Deep Neural Networks
• Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
• Maximum-Margin Structured Learning with Deep Networks for 3D Human Pose Estimation

Cited by in corpus (11)

• A simple yet effective baseline for 3d human pose estimation
• Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views
• 3D Human Pose Estimation in the Wild by Adversarial Learning
• Learning to Estimate 3D Human Pose and Shape from a Single Color Image
• Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision
• Integral Human Pose Regression
• DeepSkeleton: Skeleton Map for 3D Human Pose Regression
• Ordinal Depth Supervision for 3D Human Pose Estimation
• Dense 3D Regression for Hand Pose Estimation
• Rethinking Pose in 3D: Multi-stage Refinement and Recovery for Markerless Motion Capture
• Explicit Spatiotemporal Joint Relation Learning for Tracking Human Pose