General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues

Abstract

Markerless motion capture algorithms require a 3D body with properly personalized skeleton dimension and/or body shape and appearance to successfully track a person. Unfortunately, many tracking methods consider model personalization a different problem and use manual or semi-automatic model initialization, which greatly reduces applicability. In this paper, we propose a fully automatic algorithm that jointly creates a rigged actor model commonly used for animation - skeleton, volumetric shape, appearance, and optionally a body surface - and estimates the actor's motion from multi-view video input only. The approach is rigorously designed to work on footage of general outdoor scenes recorded with very few cameras and without background subtraction. Our method uses a new image formation model with analytic visibility and analytically differentiable alignment energy. For reconstruction, 3D body shape is approximated as Gaussian density field. For pose and shape estimation, we minimize a new edge-based alignment energy inspired by volume raycasting in an absorbing medium. We further propose a new statistical human body model that represents the body surface, volumetric Gaussian density, as well as variability in skeleton shape. Given any multi-view sequence, our method jointly optimizes the pose and shape parameters of this model fully automatically in a spatiotemporal way.

Accepted to ECCV 2016, added additional references

References in corpus (2)

• Building Statistical Shape Spaces for 3D Human Modeling
• General Dynamic Scene Reconstruction from Multiple View Video

Cited by in corpus (12)

• BodyNet: Volumetric Inference of 3D Human Body Shapes
• MonoPerfCap: Human Performance Capture from Monocular Video
• Video Based Reconstruction of 3D People Models
• Detailed, accurate, human shape estimation from clothed 3D scan sequences
• Learning to Estimate 3D Human Pose and Shape from a Single Color Image
• DeepCap: Monocular Human Performance Capture Using Weak Supervision
• Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision
• EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras
• Model-based Outdoor Performance Capture
• Deep Reinforcement Learning for Active Human Pose Estimation
• Soccer on Your Tabletop
• Robust 3D Human Motion Reconstruction Via Dynamic Template Construction