A Survey and Taxonomy of Adversarial Neural Networks for Text-to-Image Synthesis
arXiv:1910.09399
Abstract
Text-to-image synthesis refers to computational methods which translate human written textual descriptions, in the form of keywords or sentences, into images with similar semantic meaning to the text. In earlier research, image synthesis relied mainly on word to image correlation analysis combined with supervised methods to find best alignment of the visual content matching to the text. Recent progress in deep learning (DL) has brought a new set of unsupervised deep learning methods, particularly deep generative models which are able to generate realistic visual images using suitably trained neural network models. In this paper, we review the most recent development in the text-to-image synthesis research domain. Our survey first introduces image synthesis and its challenges, and then reviews key concepts such as generative adversarial networks (GANs) and deep convolutional encoder-decoder neural networks (DCNN). After that, we propose a taxonomy to summarize GAN based text-to-image synthesis into four major categories: Semantic Enhancement GANs, Resolution Enhancement GANs, Diversity Enhancement GANS, and Motion Enhancement GANs. We elaborate the main objective of each group, and further review typical GAN architectures in each group. The taxonomy and the review outline the techniques and the evolution of different approaches, and eventually provide a clear roadmap to summarize the list of contemporaneous solutions that utilize GANs and DCNNs to generate enthralling results in categories such as human faces, birds, flowers, room interiors, object reconstruction from edge maps (games) etc. The survey will conclude with a comparison of the proposed solutions, challenges that remain unresolved, and future developments in the text-to-image synthesis domain.
arXiv commentWiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. 2019
References in corpus (19)
- Conditional Generative Adversarial Nets
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Generative Adversarial Networks: An Overview
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Generative Adversarial Text to Image Synthesis
- Conditional Image Synthesis With Auxiliary Classifier GANs
- Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
- Improved Techniques for Training GANs
- How Generative Adversarial Networks and Their Variants Work: An Overview
- Learning What and Where to Draw
- LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation
- An Introduction to Image Synthesis with Generative Adversarial Nets
- ObamaNet: Photo-realistic lip-sync from text
- MirrorGAN: Learning Text-to-image Generation by Redescription
- MC-GAN: Multi-conditional Generative Adversarial Network for Image Synthesis
- Semantic Image Synthesis via Adversarial Learning
- Semi-Latent GAN: Learning to generate and modify facial images from attributes
- I2T2I: Learning Text to Image Synthesis with Textual Data Augmentation
- Adversarial nets with perceptual losses for text-to-image synthesis