Human Image Generation: A Comprehensive Survey

Abstract

Image and video synthesis has become a blooming topic in computer vision and machine learning communities along with the developments of deep generative models, due to its great academic and application value. Many researchers have been devoted to synthesizing high-fidelity human images as one of the most commonly seen object categories in daily lives, where a large number of studies are performed based on various models, task settings and applications. Thus, it is necessary to give a comprehensive overview on these variant methods on human image generation. In this paper, we divide human image generation techniques into three paradigms, i.e., data-driven methods, knowledge-guided methods and hybrid methods. For each paradigm, the most representative models and the corresponding variants are presented, where the advantages and characteristics of different methods are summarized in terms of model architectures. Besides, the main public human image datasets and evaluation metrics in the literature are summarized. Furthermore, due to the wide application potentials, the typical downstream usages of synthesized human images are covered. Finally, the challenges and potential opportunities of human image generation are discussed to shed light on future research.

Accepted by ACM Computing Surveys (CSUR)

References in corpus (33)

• Denoising Diffusion Probabilistic Models
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• A Survey on Visual Transformer
• Transformers in Vision: A Survey
• VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera
• Learning from Synthetic Humans
• A note on the evaluation of generative models
• GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
• Looking Beyond Appearances: Synthetic Training Data for Deep CNNs in Re-identification
• MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild
• STAR: Sparse Trained Articulated Human Body Regressor
• StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
• NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)
• Deep Spatial Transformation for Pose-Guided Person Image Generation and Animation
• Pedestrian-Synthesis-GAN: Generating Pedestrian Data in Real Scene and Beyond
• PoNA: Pose-guided Non-local Attention for Human Pose Transfer
• Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control
• Towards Fine-grained Human Pose Transfer with Detail Replenishing Network
• Detailed Garment Recovery from a Single-View Image
• Bipartite Graph Reasoning GANs for Person Image Generation
• Deep Person Generation: A Survey from the Perspective of Face, Pose and Cloth Synthesis
• Towards Scalable Unpaired Virtual Try-On via Patch-Routed Spatially-Adaptive GAN
• Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models
• HumanDiffusion: a Coarse-to-Fine Alignment Diffusion Framework for Controllable Text-Driven Person Image Generation
• Image Comes Dancing with Collaborative Parsing-Flow Video Synthesis
• DwNet: Dense warp-based network for pose-guided human video generation
• Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models
• AG3D: Learning to Generate 3D Avatars from 2D Image Collections
• LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-On
• Text2Performer: Text-Driven Human Video Generation
• ActorsNeRF: Animatable Few-shot Human Rendering with Generalizable NeRFs
• Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editing
• UPGPT: Universal Diffusion Model for Person Image Generation, Editing and Pose Transfer