ColdGANs: Taming Language GANs with Cautious Sampling Strategies

Abstract

Training regimes based on Maximum Likelihood Estimation (MLE) suffer from known limitations, often leading to poorly generated text sequences. At the root of these limitations is the mismatch between training and inference, i.e. the so-called exposure bias, exacerbated by considering only the reference texts as correct, while in practice several alternative formulations could be as good. Generative Adversarial Networks (GANs) can mitigate those limitations but the discrete nature of text has hindered their application to language generation: the approaches proposed so far, based on Reinforcement Learning, have been shown to underperform MLE. Departing from previous works, we analyze the exploration step in GANs applied to text generation, and show how classical sampling results in unstable training. We propose to consider alternative exploration strategies in a GAN framework that we name ColdGANs, where we force the sampling to be close to the distribution modes to get smoother learning dynamics. For the first time, to the best of our knowledge, the proposed language GANs compare favorably to MLE, and obtain improvements over the state-of-the-art on three generative tasks, namely unconditional text generation, question generation, and abstractive summarization.

References in corpus (29)

• Distilling the Knowledge in a Neural Network
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• A Deep Reinforced Model for Abstractive Summarization
• The Curious Case of Neural Text Degeneration
• Sequence Level Training with Recurrent Neural Networks
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
• Fine-Tuning Language Models from Human Preferences
• Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
• MaskGAN: Better Text Generation via Filling in the______
• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
• Neural Text Generation with Unlikelihood Training
• Adversarial Learning for Neural Dialogue Generation
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Long Text Generation via Adversarial Training with Leaked Information
• Texygen: A Benchmarking Platform for Text Generation Models
• Text Summarization with Pretrained Encoders
• Adversarial examples from computational constraints
• Learning to Ask: Neural Question Generation for Reading Comprehension
• Defending Against Neural Fake News
• On Accurate Evaluation of GANs for Language Generation
• Adversarial Spheres
• Neural Question Generation from Text: A Preliminary Study
• Evaluating Text GANs as Language Models
• Learning by Semantic Similarity Makes Abstractive Summarization Better
• PyText: A Seamless Path from NLP research to production
• Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering
• Discriminative Adversarial Search for Abstractive Summarization

Cited by in corpus (4)

• Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges
• Unpacking the Expressed Consequences of AI Research in Broader Impact Statements
• Residual Energy-Based Models for Text
• Rethinking Automatic Evaluation in Sentence Simplification