ALR-GAN: Adaptive Layout Refinement for Text-to-Image Synthesis

Abstract

We propose a novel Text-to-Image Generation Network, Adaptive Layout Refinement Generative Adversarial Network (ALR-GAN), to adaptively refine the layout of synthesized images without any auxiliary information. The ALR-GAN includes an Adaptive Layout Refinement (ALR) module and a Layout Visual Refinement (LVR) loss. The ALR module aligns the layout structure (which refers to locations of objects and background) of a synthesized image with that of its corresponding real image. In ALR module, we proposed an Adaptive Layout Refinement (ALR) loss to balance the matching of hard and easy features, for more efficient layout structure matching. Based on the refined layout structure, the LVR loss further refines the visual representation within the layout area. Experimental results on two widely-used datasets show that ALR-GAN performs competitively at the Text-to-Image generation task.

arXiv commentAccepted by TMM

References in corpus (9)

• Spectral Normalization for Generative Adversarial Networks
• Generative Adversarial Text to Image Synthesis
• Dual-Path Convolutional Image-Text Embeddings with Instance Loss
• Semantic Object Accuracy for Generative Text-to-Image Synthesis
• ChatPainter: Improving Text to Image Generation using Dialogue
• PasteGAN: A Semi-Parametric Method to Generate Image from Scene Graph
• Text-Guided Neural Image Inpainting
• Describe What to Change: A Text-guided Unsupervised Image-to-Image Translation Approach
• Leveraging Visual Question Answering to Improve Text-to-Image Synthesis