How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
arXiv:1511.05101
Abstract
Modern applications and progress in deep learning research have created renewed interest for generative models of text and of images. However, even today it is unclear what objective functions one should use to train and evaluate these models. In this paper we present two contributions. Firstly, we present a critique of scheduled sampling, a state-of-the-art training method that contributed to the winning entry to the MSCOCO image captioning benchmark in 2015. Here we show that despite this impressive empirical performance, the objective function underlying scheduled sampling is improper and leads to an inconsistent learning algorithm. Secondly, we revisit the problems that scheduled sampling was meant to address, and present an alternative interpretation. We argue that maximum likelihood is an inappropriate training objective when the end-goal is to generate natural-looking samples. We go on to derive an ideal objective function to use in this situation instead. We introduce a generalisation of adversarial training, and show how such method can interpolate between maximum likelihood training and our ideal training objective. To our knowledge this is the first theoretical analysis that explains why adversarial training tends to produce samples with higher perceived quality.
References in corpus (5)
Cited by in corpus (64)
- Deep Learning for Single Image Super-Resolution: A Brief Review
- f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
- Improved Image Captioning via Policy Gradient optimization of SPIDEr
- Professor Forcing: A New Algorithm for Training Recurrent Networks
- SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
- GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
- Learning in Implicit Generative Models
- Texygen: A Benchmarking Platform for Text Generation Models
- Adversarial Feature Matching for Text Generation
- Dual Discriminator Generative Adversarial Nets
- Machine Learning for Spatiotemporal Sequence Forecasting: A Survey
- Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models
- Stabilizing Generative Adversarial Networks: A Survey
- Wasserstein Learning of Deep Generative Point Process Models
- Implicit Maximum Likelihood Estimation
- Neural Text Generation: Past, Present and Beyond
- Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network
- Multi-Generator Generative Adversarial Nets
- Generative Adversarial Nets from a Density Ratio Estimation Perspective
- Yes, we GAN: Applying Adversarial Techniques for Autonomous Driving
- Learning Causal Semantic Representation for Out-of-Distribution Prediction
- Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
- Regularizing RNNs for Caption Generation by Reconstructing The Past with The Present
- Improving Sequence-to-Sequence Learning via Optimal Transport
- Neural Language Generation: Formulation, Methods, and Evaluation
- BFGAN: Backward and Forward Generative Adversarial Networks for Lexically Constrained Sentence Generation
- On the Effectiveness of Least Squares Generative Adversarial Networks
- Introduction to Normalizing Flows for Lattice Field Theory
- On the Implicit Assumptions of GANs
- Task Specific Adversarial Cost Function
- Topic-Preserving Synthetic News Generation: An Adversarial Deep Reinforcement Learning Approach
- Visual Time Series Forecasting: An Image-driven Approach
- A New GAN-based End-to-End TTS Training Algorithm
- Learning to Match Distributions for Domain Adaptation
- Rethinking Exposure Bias In Language Modeling
- Delving Deeper into the Decoder for Video Captioning
- CatGAN: Category-aware Generative Adversarial Networks with Hierarchical Evolutionary Learning for Category Text Generation
- Generating Music with a Self-Correcting Non-Chronological Autoregressive Model
- Rescuing neural spike train models from bad MLE
- Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation
- Text Generation by Learning from Demonstrations
- Photon detection probability prediction using one-dimensional generative neural network
- Sliced Iterative Normalizing Flows
- Text Generation Based on Generative Adversarial Nets with Latent Variable
- Goal-directed Generation of Discrete Structures with Conditional Generative Models
- Adding A Filter Based on The Discriminator to Improve Unconditional Text Generation
- Generative Adversarial Networks are Special Cases of Artificial Curiosity (1990) and also Closely Related to Predictability Minimization (1991)
- Teacher-Student Training for Robust Tacotron-based TTS
- On Value Discrepancy of Imitation Learning
- Controllable Dual Skew Divergence Loss for Neural Machine Translation
- Generative Adversarial Networks and Adversarial Autoencoders: Tutorial and Survey
- Language Model Evaluation in Open-ended Text Generation
- Adversarial Self-Supervised Data-Free Distillation for Text Classification
- Learning to Generate 3D Shapes with Generative Cellular Automata
- -Neighbor Based Curriculum Sampling for Sequence Prediction
- Leveraging GPT-2 for Classifying Spam Reviews with Limited Labeled Data via Adversarial Training
- A study of traits that affect learnability in GANs
- Stylized Text Generation Using Wasserstein Autoencoders with a Mixture of Gaussian Prior
- Safety Aware Reinforcement Learning (SARL)
- Diversifying Topic-Coherent Response Generation for Natural Multi-turn Conversations
- HpGAN: Sequence Search with Generative Adversarial Networks
- Efficient text generation of user-defined topic using generative adversarial networks
- Melody-Conditioned Lyrics Generation with SeqGANs
- Autoregressive Knowledge Distillation through Imitation Learning