Rethinking Pose in 3D: Multi-stage Refinement and Recovery for Markerless Motion Capture

Abstract

We propose a CNN-based approach for multi-camera markerless motion capture of the human body. Unlike existing methods that first perform pose estimation on individual cameras and generate 3D models as post-processing, our approach makes use of 3D reasoning throughout a multi-stage approach. This novelty allows us to use provisional 3D models of human pose to rethink where the joints should be located in the image and to recover from past mistakes. Our principled refinement of 3D human poses lets us make use of image cues, even from images where we previously misdetected joints, to refine our estimates as part of an end-to-end approach. Finally, we demonstrate how the high-quality output of our multi-camera setup can be used as an additional training source to improve the accuracy of existing single camera models.

arXiv commentInternational Conference on 3DVision (3dv)

References in corpus (14)

• Learning from Synthetic Humans
• Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image
• Stacked Hourglass Networks for Human Pose Estimation
• Convolutional Pose Machines
• MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild
• A simple yet effective baseline for 3d human pose estimation
• Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach
• Compositional Human Pose Regression
• Structured Prediction of 3D Human Pose with Deep Neural Networks
• Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
• DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation
• Predicting People's 3D Poses from Short Sequences
• Learning to Fuse 2D and 3D Image Cues for Monocular Body Pose Estimation
• 3D Human Pose Estimation from a Single Image via Distance Matrix Regression