3D Human Pose, Shape and Texture from Low-Resolution Images and Videos

Abstract

3D human pose and shape estimation from monocular images has been an active research area in computer vision. Existing deep learning methods for this task rely on high-resolution input, which however, is not always available in many scenarios such as video surveillance and sports broadcasting. Two common approaches to deal with low-resolution images are applying super-resolution techniques to the input, which may result in unpleasant artifacts, or simply training one model for each resolution, which is impractical in many realistic applications. To address the above issues, this paper proposes a novel algorithm called RSC-Net, which consists of a Resolution-aware network, a Self-supervision loss, and a Contrastive learning scheme. The proposed method is able to learn 3D body pose and shape across different resolutions with one single model. The self-supervision loss enforces scale-consistency of the output, and the contrastive learning scheme enforces scale-consistency of the deep features. We show that both these new losses provide robustness when learning in a weakly-supervised manner. Moreover, we extend the RSC-Net to handle low-resolution videos and apply it to reconstruct textured 3D pedestrians from low-resolution input. Extensive experiments demonstrate that the RSC-Net can achieve consistently better results than the state-of-the-art methods for challenging low-resolution images.

arXiv commentarXiv admin note: substantial text overlap with arXiv:2007.13666

References in corpus (11)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Distilling the Knowledge in a Neural Network
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Temporal Ensembling for Semi-Supervised Learning
• Learning from Synthetic Humans
• Contrastive Multiview Coding
• Low-resolution Face Recognition in the Wild via Selective Knowledge Distillation
• ReZero is All You Need: Fast Convergence at Large Depth
• PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
• Sim2real transfer learning for 3D human pose estimation: motion to the rescue