Maximum-Likelihood Augmented Discrete Generative Adversarial Networks

Abstract

Despite the successes in capturing continuous distributions, the application of generative adversarial networks (GANs) to discrete settings, like natural language tasks, is rather restricted. The fundamental reason is the difficulty of back-propagation through discrete random variables combined with the inherent instability of the GAN training objective. To address these problems, we propose Maximum-Likelihood Augmented Discrete Generative Adversarial Networks. Instead of directly optimizing the GAN objective, we derive a novel and low-variance objective using the discriminator's output that follows corresponds to the log-likelihood. Compared with the original, the new objective is proved to be consistent in theory and beneficial in practice. The experimental results on various discrete datasets demonstrate the effectiveness of the proposed approach.

11 pages, 3 figures

References in corpus (5)

• Conditional Generative Adversarial Nets
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Mode Regularized Generative Adversarial Networks
• Reward Augmented Maximum Likelihood for Neural Structured Prediction

Cited by in corpus (17)

• Deep learning for molecular design - a review of the state of the art
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Long Text Generation via Adversarial Training with Leaked Information
• Generating and designing DNA with deep generative models
• Adversarial Generation of Natural Language
• Language Generation with Recurrent Generative Adversarial Networks without Pre-training
• Unsupervised Cipher Cracking Using Discrete GANs
• Neural Language Generation: Formulation, Methods, and Evaluation
• Justifying Diagnosis Decisions by Deep Neural Networks
• Self-Adversarial Learning with Comparative Discrimination for Text Generation
• Jointly Measuring Diversity and Quality in Text Generation Models
• Weakly-supervised Knowledge Graph Alignment with Adversarial Learning
• Adversarial Sub-sequence for Text Generation
• Sample weighting as an explanation for mode collapse in generative adversarial networks
• Classification of sparsely labeled spatio-temporal data through semi-supervised adversarial learning
• The Detection of Distributional Discrepancy for Text Generation
• Evaluating Computational Language Models with Scaling Properties of Natural Language