TAC-GAN - Text Conditioned Auxiliary Classifier Generative Adversarial Network

Abstract

In this work, we present the Text Conditioned Auxiliary Classifier Generative Adversarial Network, (TAC-GAN) a text to image Generative Adversarial Network (GAN) for synthesizing images from their text descriptions. Former approaches have tried to condition the generative process on the textual data; but allying it to the usage of class information, known to diversify the generated samples and improve their structural coherence, has not been explored. We trained the presented TAC-GAN model on the Oxford-102 dataset of flowers, and evaluated the discriminability of the generated images with Inception-Score, as well as their diversity using the Multi-Scale Structural Similarity Index (MS-SSIM). Our approach outperforms the state-of-the-art models, i.e., its inception score is 3.45, corresponding to a relative increase of 7.8% compared to the recently introduced StackGan. A comparison of the mean MS-SSIM scores of the training and generated samples per class shows that our approach is able to generate highly diverse images with an average MS-SSIM of 0.14 over all generated classes.

References in corpus (7)

• Auto-Encoding Variational Bayes
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Tutorial on Variational Autoencoders
• DRAW: A Recurrent Neural Network For Image Generation

Cited by in corpus (22)

• Multimodal Intelligence: Representation Learning, Information Fusion, and Applications
• How Generative Adversarial Networks and Their Variants Work: An Overview
• Adversarial Text-to-Image Synthesis: A Review
• An Introduction to Image Synthesis with Generative Adversarial Nets
• Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
• FakeNews: GAN-based generation of realistic 3D volumetric data -- A systematic review and taxonomy
• DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis
• Vector Quantized Diffusion Model for Text-to-Image Synthesis
• A Decentralized Parallel Algorithm for Training Generative Adversarial Nets
• Text-to-Image-to-Text Translation using Cycle Consistent Adversarial Networks
• Adversarial symmetric GANs: bridging adversarial samples and adversarial networks
• on the effectiveness of generative adversarial network on anomaly detection
• Leveraging Visual Question Answering to Improve Text-to-Image Synthesis
• Latent Normalizing Flows for Many-to-Many Cross-Domain Mappings
• Dual Adversarial Inference for Text-to-Image Synthesis
• Level generation and style enhancement -- deep learning for game development overview
• Ear2Face: Deep Biometric Modality Mapping
• MemeFaceGenerator: Adversarial Synthesis of Chinese Meme-face from Natural Sentences
• Weakly-supervised Generative Adversarial Networks for medical image classification
• Using General Adversarial Networks for Marketing: A Case Study of Airbnb
• Improving Text to Image Generation using Mode-seeking Function
• Words as Art Materials: Generating Paintings with Sequential GANs