Be Your Own Prada: Fashion Synthesis with Structural Coherence

Abstract

We present a novel and effective approach for generating new clothing on a wearer through generative adversarial learning. Given an input image of a person and a sentence describing a different outfit, our model "redresses" the person as desired, while at the same time keeping the wearer and her/his pose unchanged. Generating new outfits with precise regions conforming to a language description while retaining wearer's body structure is a new challenging task. Existing generative adversarial networks are not ideal in ensuring global coherence of structure given both the input photograph and language description as conditions. We address this challenge by decomposing the complex generative process into two conditional stages. In the first stage, we generate a plausible semantic segmentation map that obeys the wearer's pose as a latent spatial arrangement. An effective spatial constraint is formulated to guide the generation of this semantic segmentation map. In the second stage, a generative model with a newly proposed compositional mapping layer is used to render the final image with precise regions and textures conditioned on this map. We extended the DeepFashion dataset [8] by collecting sentence descriptions for 79K images. We demonstrate the effectiveness of our approach through both quantitative and qualitative evaluations. A user study is also conducted. The codes and the data are available at http://mmlab.ie.cuhk. edu.hk/projects/FashionGAN/.

arXiv commentThis is the updated version of our original paper appeared in ICCV 2017 proceedings

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Improved Techniques for Training GANs
• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
• Contextual Action Recognition with R*CNN
• Detailed Garment Recovery from a Single-View Image

Cited by in corpus (18)

• Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform
• Fashion-Gen: The Generative Fashion Dataset and Challenge
• Fashion Meets Computer Vision: A Survey
• Learning Type-Aware Embeddings for Fashion Compatibility
• Unsupervised Person Image Synthesis in Arbitrary Poses
• Disentangling Multiple Conditional Inputs in GANs
• Toward Characteristic-Preserving Image-based Virtual Try-On Network
• Pose Guided Fashion Image Synthesis Using Deep Generative Model
• Sequential Attention GAN for Interactive Image Editing
• Learning Compositional Neural Information Fusion for Human Parsing
• Learning to Synthesize Fashion Textures
• Apparel-invariant Feature Learning for Apparel-changed Person Re-identification
• CRAFT: Complementary Recommendations Using Adversarial Feature Transformer
• Generating Interactive Worlds with Text
• Parametric Synthesis of Text on Stylized Backgrounds using PGGANs
• A Pragmatic AI Approach to Creating Artistic Visual Variations by Neural Style Transfer
• CreativeBioMan: Brain and Body Wearable Computing based Creative Gaming System
• Image-based Virtual Fitting Room