VQ-HPS: Human Pose and Shape Estimation in a Vector-Quantized Latent Space

Abstract

Previous works on Human Pose and Shape Estimation (HPSE) from RGB images can be broadly categorized into two main groups: parametric and non-parametric approaches. Parametric techniques leverage a low-dimensional statistical body model for realistic results, whereas recent non-parametric methods achieve higher precision by directly regressing the 3D coordinates of the human body mesh. This work introduces a novel paradigm to address the HPSE problem, involving a low-dimensional discrete latent representation of the human mesh and framing HPSE as a classification task. Instead of predicting body model parameters or 3D vertex coordinates, we focus on predicting the proposed discrete latent representation, which can be decoded into a registered human mesh. This innovative paradigm offers two key advantages. Firstly, predicting a low-dimensional discrete representation confines our predictions to the space of anthropomorphic poses and shapes even when little training data is available. Secondly, by framing the problem as a classification task, we can harness the discriminative power inherent in neural networks. The proposed model, VQ-HPS, predicts the discrete latent representation of the mesh. The experimental results demonstrate that VQ-HPS outperforms the current state-of-the-art non-parametric approaches while yielding results as realistic as those produced by parametric methods when trained with little data. VQ-HPS also shows promising results when training on large-scale datasets, highlighting the significant potential of the classification approach for HPSE. See the project page at https://g-fiche.github.io/research-pages/vqhps/

References in corpus (15)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Neural Discrete Representation Learning
• Learning from Synthetic Humans
• LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Images
• MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild
• Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernels
• Synthetic Training for Accurate 3D Human Pose and Shape Estimation in the Wild
• SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation
• Playing for 3D Human Recovery
• Multi-Person Extreme Motion Prediction
• Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithms
• Learning 3D Human Shape and Pose from Dense Body Parts
• ChatPose: Chatting about 3D Human Pose
• TORE: Token Reduction for Efficient Human Mesh Recovery with Transformer
• Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstruction