Adversarial Text Generation Without Reinforcement Learning

Abstract

Generative Adversarial Networks (GANs) have experienced a recent surge in popularity, performing competitively in a variety of tasks, especially in computer vision. However, GAN training has shown limited success in natural language processing. This is largely because sequences of text are discrete, and thus gradients cannot propagate from the discriminator to the generator. Recent solutions use reinforcement learning to propagate approximate gradients to the generator, but this is inefficient to train. We propose to utilize an autoencoder to learn a low-dimensional representation of sentences. A GAN is then trained to generate its own vectors in this space, which decode to realistic utterances. We report both random and interpolated samples from the generator. Visualization of sentence vectors indicate our model correctly learns the latent space of the autoencoder. Both human ratings and BLEU scores show that our model generates realistic text against competitive baselines.

arXiv commentFour pages without references. ACL latex style. Four figures

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Improved Training of Wasserstein GANs
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
• Adversarial Learning for Neural Dialogue Generation
• Adversarial Feature Matching for Text Generation
• Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
• Making Neural QA as Simple as Possible but not Simpler

Cited by in corpus (3)

• AutoML: A Survey of the State-of-the-Art
• A Deep Learning Framework for Generation and Analysis of Driving Scenario Trajectories
• Claim Verification using a Multi-GAN based Model