Monocular Human Shape and Pose with Dense Mesh-borne Local Image Features

Abstract

We propose to improve on graph convolution based approaches for human shape and pose estimation from monocular input, using pixel-aligned local image features. Given a single input color image, existing graph convolutional network (GCN) based techniques for human shape and pose estimation use a single convolutional neural network (CNN) generated global image feature appended to all mesh vertices equally to initialize the GCN stage, which transforms a template T-posed mesh into the target pose. In contrast, we propose for the first time the idea of using local image features per vertex. These features are sampled from the CNN image feature maps by utilizing pixel-to-mesh correspondences generated with DensePose. Our quantitative and qualitative results on standard benchmarks show that using local features improves on global ones and leads to competitive performances with respect to the state-of-the-art.

arXiv commentFG 2021

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Semi-Supervised Classification with Graph Convolutional Networks
• Semantic Graph Convolutional Networks for 3D Human Pose Regression
• Self-supervised Learning of Motion Capture
• BodyNet: Volumetric Inference of 3D Human Body Shapes
• Single Image 3D Hand Reconstruction with Mesh Convolutions
• Cross-modal Deep Face Normals with Deactivable Skip Connections
• Learning 3D Human Shape and Pose from Dense Body Parts