Bilinear Representation for Language-based Image Editing Using Conditional Generative Adversarial Networks

Abstract

The task of Language-Based Image Editing (LBIE) aims at generating a target image by editing the source image based on the given language description. The main challenge of LBIE is to disentangle the semantics in image and text and then combine them to generate realistic images. Therefore, the editing performance is heavily dependent on the learned representation. In this work, conditional generative adversarial network (cGAN) is utilized for LBIE. We find that existing conditioning methods in cGAN lack of representation power as they cannot learn the second-order correlation between two conditioning vectors. To solve this problem, we propose an improved conditional layer named Bilinear Residual Layer (BRL) to learning more powerful representations for LBIE task. Qualitative and quantitative comparisons demonstrate that our method can generate images with higher quality when compared to previous LBIE techniques.

To appear at ICASSP 2019. Implementation: https://github.com/vtddggg/BilinearGAN_for_LBIE

References in corpus (5)

• Conditional Generative Adversarial Nets
• WaveNet: A Generative Model for Raw Audio
• Conditional Image Generation with PixelCNN Decoders
• Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
• Language Guided Fashion Image Manipulation with Feature-wise Transformations

Cited by in corpus (7)

• Deep Learning for Joint Channel Estimation and Signal Detection in OFDM Systems
• Deep Learning-Aided 6G Wireless Networks: A Comprehensive Survey of Revolutionary PHY Architectures
• GAP++: Learning to generate target-conditioned adversarial examples
• Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions
• Text2Poster: Laying out Stylized Texts on Retrieved Images
• SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Editing
• Learning by Planning: Language-Guided Global Image Editing