Language Generation with Recurrent Generative Adversarial Networks without Pre-training

Abstract

Generative Adversarial Networks (GANs) have shown great promise recently in image generation. Training GANs for language generation has proven to be more difficult, because of the non-differentiable nature of generating text with recurrent neural networks. Consequently, past work has either resorted to pre-training with maximum-likelihood or used convolutional networks for generation. In this work, we show that recurrent neural networks can be trained to generate text with GANs from scratch using curriculum learning, by slowly teaching the model to generate sequences of increasing and variable length. We empirically show that our approach vastly improves the quality of generated sequences compared to a convolutional baseline.

Presented at the 1st Workshop on Learning to Generate Natural Language at ICML 2017

References in corpus (3)

• Conditional Generative Adversarial Nets
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Recurrent Topic-Transition GAN for Visual Paragraph Generation

Cited by in corpus (5)

• LD-SDS: Towards an Expressive Spoken Dialogue System based on Linked-Data
• Pipeline Generative Adversarial Networks for Facial Images Generation with Multiple Attributes
• Classification of sparsely labeled spatio-temporal data through semi-supervised adversarial learning
• Instance Map based Image Synthesis with a Denoising Generative Adversarial Network
• Generative Adversarial Nets for Multiple Text Corpora