Pose Guided Person Image Generation

Abstract

This paper proposes the novel Pose Guided Person Generation Network (PG${}^2$) that allows to synthesize person images in arbitrary poses, based on an image of that person and a novel pose. Our generation framework PG${}^2$ utilizes the pose information explicitly and consists of two key stages: pose integration and image refinement. In the first stage the condition image and the target pose are fed into a U-Net-like network to generate an initial but coarse image of the person with the target pose. The second stage then refines the initial and blurry result by training a U-Net-like generator in an adversarial way. Extensive experimental results on both 128$\times$64 re-identification images and 256$\times$256 fashion photos show that our model generates high-quality person images with convincing details.

Xu Jia and Qianru Sun contribute equally. Accepted in Proceedings of 31st Conference on Neural Information Processing Systems (NIPS 2017)

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Improved Techniques for Training GANs
• Stacked Hourglass Networks for Human Pose Estimation
• FusionNet: A deep fully residual convolutional neural network for image segmentation in connectomics
• Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
• Stacked Generative Adversarial Networks

Cited by in corpus (95)

• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics
• DRIT++: Diverse Image-to-Image Translation via Disentangled Representations
• HeadOn: Real-time Reenactment of Human Portrait Videos
• Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics
• Joint Discriminative and Generative Learning for Person Re-identification
• Deep Spatial Transformation for Pose-Guided Person Image Generation and Animation
• Pedestrian-Synthesis-GAN: Generating Pedestrian Data in Real Scene and Beyond
• Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control
• Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation
• Fashion Meets Computer Vision: A Survey
• Towards Fine-grained Human Pose Transfer with Detail Replenishing Network
• Soft-Gated Warping-GAN for Pose-Guided Person Image Synthesis
• Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
• Language Guided Fashion Image Manipulation with Feature-wise Transformations
• Distinguishing Natural and Computer-Generated Images using Multi-Colorspace fused EfficientNet
• Pose-Normalized Image Generation for Person Re-identification
• Text as Neural Operator: Image Manipulation by Text Instruction
• One-shot Face Reenactment
• Multi-Channel Attention Selection GANs for Guided Image-to-Image Translation
• Deep Image Spatial Transformation for Person Image Generation
• Generative Adversarial Networks: A Survey Towards Private and Secure Applications
• MetaPix: Few-Shot Video Retargeting
• Controllable Person Image Synthesis with Attribute-Decomposed GAN
• Human Motion Transfer with 3D Constraints and Detail Enhancement
• Disentangled Person Image Generation
• M2E-Try On Net: Fashion from Model to Everyone
• DanceIt: Music-inspired Dancing Video Synthesis
• Towards Visually Explaining Variational Autoencoders
• Image-to-Image Translation: Methods and Applications
• COVID-19 CT Image Synthesis with a Conditional Generative Adversarial Network
• Unsupervised Person Image Synthesis in Arbitrary Poses
• Gesture-to-Gesture Translation in the Wild via Category-Independent Conditional Maps
• Triple consistency loss for pairing distributions in GAN-based face synthesis
• Toward Characteristic-Preserving Image-based Virtual Try-On Network
• Neural Rendering and Reenactment of Human Actor Videos
• Vision and Language: from Visual Perception to Content Creation
• Coordinate-based Texture Inpainting for Pose-Guided Image Generation
• High Fidelity Face Manipulation with Extreme Poses and Expressions
• How does Lipschitz Regularization Influence GAN Training?
• Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation
• Transformable Bottleneck Networks
• Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks
• Learning to Forecast and Refine Residual Motion for Image-to-Video Generation
• Pose Guided Fashion Image Synthesis Using Deep Generative Model
• Video synthesis of human upper body with realistic face
• What Is It Like Down There? Generating Dense Ground-Level Views and Image Features From Overhead Imagery Using Conditional Generative Adversarial Networks
• TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting
• Recapture as You Want
• Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization
• Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter Corruption
• GestureGAN for Hand Gesture-to-Gesture Translation in the Wild
• FReeNet: Multi-Identity Face Reenactment
• Pose-Guided High-Resolution Appearance Transfer via Progressive Training
• Robust Pose Transfer with Dynamic Details using Neural Video Rendering
• Make a Face: Towards Arbitrary High Fidelity Face Manipulation
• Asymmetric GANs for Image-to-Image Translation
• Bi-level Feature Alignment for Versatile Image Translation and Manipulation
• Dense Pose Transfer
• Dance Generation with Style Embedding: Learning and Transferring Latent Representations of Dance Styles
• SPG-VTON: Semantic Prediction Guidance for Multi-pose Virtual Try-on
• Disentangling Content and Style via Unsupervised Geometry Distillation
• Semantically Multi-modal Image Synthesis
• Future Urban Scenes Generation Through Vehicles Synthesis
• Fashion Editing with Adversarial Parsing Learning
• Attribute-Guided Sketch Generation
• MsCGAN: Multi-scale Conditional Generative Adversarial Networks for Person Image Generation
• Image-based Virtual Fitting Room
• Natural and Effective Obfuscation by Head Inpainting
• Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies
• CoPE: Conditional image generation using Polynomial Expansions
• Combining Attention with Flow for Person Image Synthesis
• Neural Articulated Radiance Field
• Multi-Frame Content Integration with a Spatio-Temporal Attention Mechanism for Person Video Motion Transfer
• PortraitGAN for Flexible Portrait Manipulation
• Text-to-image Synthesis via Symmetrical Distillation Networks
• Sparse Pose Trajectory Completion
• Pose-Guided Sign Language Video GAN with Dynamic Lambda
• CT-Net: Complementary Transfering Network for Garment Transfer with Arbitrary Geometric Changes
• Directional GAN: A Novel Conditioning Strategy for Generative Networks
• Self-supervised Correlation Mining Network for Person Image Generation
• EgoRenderer: Rendering Human Avatars from Egocentric Camera Images
• GLocal: Global Graph Reasoning and Local Structure Transfer for Person Image Generation
• 3D-Aware Semantic-Guided Generative Model for Human Synthesis
• An Efficient Style Virtual Try on Network for Clothing Business Industry
• Channel Decomposition into Painting Actions
• Adversarial Synthesis of Human Pose from Text
• An Unpaired Shape Transforming Method for Image Translation and Cross-Domain Retrieval
• Generating a Fusion Image: One's Identity and Another's Shape
• PCGAN: Partition-Controlled Human Image Generation
• Improving Generalization via Attribute Selection on Out-of-the-box Data
• Self-Refining Deep Symmetry Enhanced Network for Rain Removal
• RIN: Textured Human Model Recovery and Imitation with a Single Image
• Pose Guided Person Image Generation with Hidden p-Norm Regression
• MGD-GAN: Text-to-Pedestrian generation through Multi-Grained Discrimination