CoT: Cooperative Training for Generative Modeling of Discrete Data

Abstract

In this paper, we study the generative models of sequential discrete data. To tackle the exposure bias problem inherent in maximum likelihood estimation (MLE), generative adversarial networks (GANs) are introduced to penalize the unrealistic generated samples. To exploit the supervision signal from the discriminator, most previous models leverage REINFORCE to address the non-differentiable problem of sequential discrete data. However, because of the unstable property of the training signal during the dynamic process of adversarial training, the effectiveness of REINFORCE, in this case, is hardly guaranteed. To deal with such a problem, we propose a novel approach called Cooperative Training (CoT) to improve the training of sequence generative models. CoT transforms the min-max game of GANs into a joint maximization framework and manages to explicitly estimate and optimize Jensen-Shannon divergence. Moreover, CoT works without the necessity of pre-training via MLE, which is crucial to the success of previous methods. In the experiments, compared to existing state-of-the-art methods, CoT shows superior or at least competitive performance on sample quality, diversity, as well as training stability.

arXiv commentAppearing as a Conference Paper on ICML 2019

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Neural Machine Translation by Jointly Learning to Align and Translate
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Instance Normalization: The Missing Ingredient for Fast Stylization
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Sequence Level Training with Recurrent Neural Networks
• Towards Principled Methods for Training Generative Adversarial Networks
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Long Text Generation via Adversarial Training with Leaked Information
• Texygen: A Benchmarking Platform for Text Generation Models
• Neural Text Generation: Past, Present and Beyond

Cited by in corpus (6)

• Language GANs Falling Short
• Re-evaluating Word Mover's Distance
• Text Generation by Learning from Demonstrations
• Follow Your Path: a Progressive Method for Knowledge Distillation
• Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation
• Evaluating Computational Language Models with Scaling Properties of Natural Language