Language Guided Fashion Image Manipulation with Feature-wise Transformations

Abstract

Developing techniques for editing an outfit image through natural sentences and accordingly generating new outfits has promising applications for art, fashion and design. However, it is considered as a certainly challenging task since image manipulation should be carried out only on the relevant parts of the image while keeping the remaining sections untouched. Moreover, this manipulation process should generate an image that is as realistic as possible. In this work, we propose FiLMedGAN, which leverages feature-wise linear modulation (FiLM) to relate and transform visual features with natural language representations without using extra spatial information. Our experiments demonstrate that this approach, when combined with skip connections and total variation regularization, produces more plausible results than the baseline work, and has a better localization capability when generating new outfits consistent with the target description.

Accepted to ECCV 2018, First Workshop on Computer Vision For Fashion, Art and Design (extended version)

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Self-Attention Generative Adversarial Networks
• A Note on the Inception Score
• DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks (with Supplementary Materials)

Cited by in corpus (4)

• Bilinear Representation for Language-based Image Editing Using Conditional Generative Adversarial Networks
• Compatible and Diverse Fashion Image Inpainting
• Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions
• Joint Visual-Textual Embedding for Multimodal Style Search