BodyNet: Volumetric Inference of 3D Human Body Shapes

Abstract

Human shape estimation is an important task for video editing, animation and fashion industry. Predicting 3D human body shape from natural images, however, is highly challenging due to factors such as variation in human bodies, clothing and viewpoint. Prior methods addressing this problem typically attempt to fit parametric body models with certain priors on pose and shape. In this work we argue for an alternative representation and propose BodyNet, a neural network for direct inference of volumetric body shape from a single image. BodyNet is an end-to-end trainable network that benefits from (i) a volumetric 3D loss, (ii) a multi-view re-projection loss, and (iii) intermediate supervision of 2D pose, 2D body part segmentation, and 3D pose. Each of them results in performance improvement as demonstrated by our experiments. To evaluate the method, we fit the SMPL model to our network output and show state-of-the-art results on the SURREAL and Unite the People datasets, outperforming recent approaches. Besides achieving state-of-the-art performance, our method also enables volumetric body-part segmentation.

arXiv commentAppears in: European Conference on Computer Vision 2018 (ECCV 2018). 27 pages

References in corpus (15)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• DeepPose: Human Pose Estimation via Deep Neural Networks
• VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera
• Learning from Synthetic Humans
• Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image
• Stacked Hourglass Networks for Human Pose Estimation
• Convolutional Pose Machines
• MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild
• Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach
• Synthesizing Training Images for Boosting Human 3D Pose Estimation
• PPFNet: Global Context Aware Local Features for Robust 3D Point Matching
• Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression
• Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image
• OctNetFusion: Learning Depth Fusion from Data
• General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues

Cited by in corpus (7)

• Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era
• MoVi: A Large Multipurpose Motion and Video Dataset
• A Neural Anthropometer Learning from Body Dimensions Computed on Human 3D Meshes
• KinePose: A temporally optimized inverse kinematics technique for 6DOF human pose estimation with biomechanical constraints
• Patch-based 3D Human Pose Refinement
• 3D Human Mesh Regression with Dense Correspondence
• Monocular Human Shape and Pose with Dense Mesh-borne Local Image Features