Neural-ABC: Neural Parametric Models for Articulated Body with Clothes

Abstract

In this paper, we introduce Neural-ABC, a novel parametric model based on neural implicit functions that can represent clothed human bodies with disentangled latent spaces for identity, clothing, shape, and pose. Traditional mesh-based representations struggle to represent articulated bodies with clothes due to the diversity of human body shapes and clothing styles, as well as the complexity of poses. Our proposed model provides a unified framework for parametric modeling, which can represent the identity, clothing, shape and pose of the clothed human body. Our proposed approach utilizes the power of neural implicit functions as the underlying representation and integrates well-designed structures to meet the necessary requirements. Specifically, we represent the underlying body as a signed distance function and clothing as an unsigned distance function, and they can be uniformly represented as unsigned distance fields. Different types of clothing do not require predefined topological structures or classifications, and can follow changes in the underlying body to fit the body. Additionally, we construct poses using a controllable articulated structure. The model is trained on both open and newly constructed datasets, and our decoupling strategy is carefully designed to ensure optimal performance. Our model excels at disentangling clothing and identity in different shape and poses while preserving the style of the clothing. We demonstrate that Neural-ABC fits new observations of different types of clothing. Compared to other state-of-the-art parametric models, Neural-ABC demonstrates powerful advantages in the reconstruction of clothed human bodies, as evidenced by fitting raw scans, depth maps and images. We show that the attributes of the fitted results can be further edited by adjusting their identities, clothing, shape and pose codes.

arXiv commentAccepted by IEEE Transactions on Visualization and Computer Graphics. Project page: https://ustc3dv.github.io/NeuralABC/

References in corpus (10)

• Embodied Hands: Modeling and Capturing Hands and Bodies Together
• STAR: Sparse Trained Articulated Human Body Regressor
• Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction
• DeepCloth: Neural Garment Representation for Shape and Style Editing
• SoftSMPL: Data-driven Modeling of Nonlinear Soft-tissue Dynamics for Parametric Humans
• Self-Correction for Human Parsing
• MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images
• Disentangled Human Body Embedding Based on Deep Hierarchical Neural Network
• MulayCap: Multi-layer Human Performance Capture Using A Monocular Video Camera
• DIG: Draping Implicit Garment over the Human Body

Cited by in corpus (1)

• GGAvatar: Reconstructing Garment-Separated 3D Gaussian Splatting Avatars from Monocular Video