3D-Aware Semantic-Guided Generative Model for Human Synthesis
arXiv:2112.01422
Abstract
Generative Neural Radiance Field (GNeRF) models, which extract implicit 3D representations from 2D images, have recently been shown to produce realistic images representing rigid/semi-rigid objects, such as human faces or cars. However, they usually struggle to generate high-quality images representing non-rigid objects, such as the human body, which is of a great interest for many computer graphics applications. This paper proposes a 3D-aware Semantic-Guided Generative Model (3D-SGAN) for human image synthesis, which combines a GNeRF with a texture generator. The former learns an implicit 3D representation of the human body and outputs a set of 2D semantic segmentation masks. The latter transforms these semantic masks into a real image, adding a realistic texture to the human appearance. Without requiring additional 3D information, our model can learn 3D human representations with a photo-realistic, controllable generation. Our experiments on the DeepFashion dataset show that 3D-SGAN significantly outperforms the most recent baselines. The code is available at https://github.com/zhangqianhui/3DSGAN
ECCV 2022. 29 pages
References in corpus (7)
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
- Alias-Free Generative Adversarial Networks
- Style and Pose Control for Image Synthesis of Humans from a Single Monocular View
- Bipartite Graph Reasoning GANs for Person Image Generation
- Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Data
- Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization