ChatPainter: Improving Text to Image Generation using Dialogue
arXiv:1802.08216
Abstract
Synthesizing realistic images from text descriptions on a dataset like Microsoft Common Objects in Context (MS COCO), where each image can contain several objects, is a challenging task. Prior work has used text captions to generate images. However, captions might not be informative enough to capture the entire image and insufficient for the model to be able to understand which objects in the images correspond to which words in the captions. We show that adding a dialogue that further describes the scene leads to significant improvement in the inception score and in the quality of generated images on the MS COCO dataset.
References in corpus (14)
- Auto-Encoding Variational Bayes
- Conditional Generative Adversarial Nets
- Spectral Normalization for Generative Adversarial Networks
- Generative Adversarial Text to Image Synthesis
- Conditional Image Synthesis With Auxiliary Classifier GANs
- Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
- Pixel Recurrent Neural Networks
- Improved Training of Wasserstein GANs
- Improved Techniques for Training GANs
- DRAW: A Recurrent Neural Network For Image Generation
- Adversarial Autoencoders
- Stabilizing Training of Generative Adversarial Networks through Regularization
- AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks
- Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis
Cited by in corpus (19)
- Multimodal Intelligence: Representation Learning, Information Fusion, and Applications
- Adversarial Text-to-Image Synthesis: A Review
- Semantic Object Accuracy for Generative Text-to-Image Synthesis
- Vector Quantized Diffusion Model for Text-to-Image Synthesis
- On the Evaluation of Conditional GANs
- ALR-GAN: Adaptive Layout Refinement for Text-to-Image Synthesis
- StoryGAN: A Sequential Conditional GAN for Story Visualization
- Cross-modal Hallucination for Few-shot Fine-grained Recognition
- IR-GAN: Image Manipulation with Linguistic Instruction by Increment Reasoning
- Adma-GAN: Attribute-Driven Memory Augmented GANs for Text-to-Image Generation
- Learning to Globally Edit Images with Textual Description
- Interactive Interior Design Recommendation via Coarse-to-fine Multimodal Reinforcement Learning
- FTGAN: A Fully-trained Generative Adversarial Networks for Text to Face Generation
- Text2Scene: Generating Compositional Scenes from Textual Descriptions
- Question-Conditioned Counterfactual Image Generation for VQA
- BlendScape: Enabling End-User Customization of Video-Conferencing Environments through Generative AI
- Image Generation from Layout
- Sequential Attention GAN for Interactive Image Editing
- Static and Animated 3D Scene Generation from Free-form Text Descriptions