Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction

Abstract

We propose Geo-PIFu, a method to recover a 3D mesh from a monocular color image of a clothed person. Our method is based on a deep implicit function-based representation to learn latent voxel features using a structure-aware 3D U-Net, to constrain the model in two ways: first, to resolve feature ambiguities in query point encoding, second, to serve as a coarse human shape proxy to regularize the high-resolution mesh and encourage global shape regularity. We show that, by both encoding query points and constraining global shape using latent voxel features, the reconstruction we obtain for clothed human meshes exhibits less shape distortion and improved surface details compared to competing methods. We evaluate Geo-PIFu on a recent human mesh public dataset that is $10\times$ larger than the private commercial dataset used in PIFu and previous derivative work. On average, we exceed the state of the art by $42.7\%$ reduction in Chamfer and Point-to-Surface Distances, and $19.4\%$ reduction in normal estimation errors.

References in corpus (6)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction
• PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization
• DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracing
• ARCH: Animatable Reconstruction of Clothed Humans

Cited by in corpus (11)

• MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images
• Deep learning for 3D human pose estimation and mesh recovery: A survey
• Neural-ABC: Neural Parametric Models for Articulated Body with Clothes
• ARCH++: Animation-Ready Clothed Human Reconstruction Revisited
• KinePose: A temporally optimized inverse kinematics technique for 6DOF human pose estimation with biomechanical constraints
• Neural Fields in Visual Computing and Beyond
• Single-view 3D Body and Cloth Reconstruction under Complex Poses
• Neural Rays for Occlusion-aware Image-based Rendering
• Reconstruction of People in Loose Clothing
• Human Performance Capture from Monocular Video in the Wild
• PeeledHuman: Robust Shape Representation for Textured 3D Human Body Reconstruction