Unsupervised Person Image Synthesis in Arbitrary Poses
arXiv:1809.10280
Abstract
We present a novel approach for synthesizing photo-realistic images of people in arbitrary poses using generative adversarial learning. Given an input image of a person and a desired pose represented by a 2D skeleton, our model renders the image of the same person under the new pose, synthesizing novel views of the parts visible in the input image and hallucinating those that are not seen. This problem has recently been addressed in a supervised manner, i.e., during training the ground truth images under the new poses are given to the network. We go beyond these approaches by proposing a fully unsupervised strategy. We tackle this challenging scenario by splitting the problem into two principal subtasks. First, we consider a pose conditioned bidirectional generator that maps back the initially rendered image to the original pose, hence being directly comparable to the input image without the need to resort to any training image. Second, we devise a novel loss function that incorporates content and style terms, and aims at producing images of high perceptual quality. Extensive experiments conducted on the DeepFashion dataset demonstrate that the images rendered by our model are very close in appearance to those obtained by fully supervised approaches.
Accepted as Spotlight at CVPR 2018
References in corpus (17)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Conditional Generative Adversarial Nets
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Generative Adversarial Text to Image Synthesis
- Conditional Image Synthesis With Auxiliary Classifier GANs
- Conditional Image Generation with PixelCNN Decoders
- Pixel Recurrent Neural Networks
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- Improved Techniques for Training GANs
- Pose Guided Person Image Generation
- Convolutional Pose Machines
- Least Squares Generative Adversarial Networks
- Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis
- Be Your Own Prada: Fashion Synthesis with Structural Coherence
- Multi-View Image Generation from a Single-View