Professor Forcing: A New Algorithm for Training Recurrent Networks

Abstract

The Teacher Forcing algorithm trains recurrent networks by supplying observed sequence values as inputs during training and using the network's own one-step-ahead predictions to do multi-step sampling. We introduce the Professor Forcing algorithm, which uses adversarial domain adaptation to encourage the dynamics of the recurrent network to be the same when training the network and when sampling from the network over multiple time steps. We apply Professor Forcing to language modeling, vocal synthesis on raw waveforms, handwriting generation, and image generation. Empirically we find that Professor Forcing acts as a regularizer, improving test likelihood on character level Penn Treebank and sequential MNIST. We also find that the model qualitatively improves samples, especially when sampling for a large number of time steps. This is supported by human evaluation of sample quality. Trade-offs between Professor Forcing and Scheduled Sampling are discussed. We produce T-SNEs showing that Professor Forcing successfully makes the dynamics of the network during training and sampling more similar.

NIPS 2016 Accepted Paper

References in corpus (8)

• Sequence to Sequence Learning with Neural Networks
• Attention-Based Models for Speech Recognition
• DRAW: A Recurrent Neural Network For Image Generation
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• MADE: Masked Autoencoder for Distribution Estimation
• Domain-Adversarial Neural Networks
• An Actor-Critic Algorithm for Sequence Prediction
• Iterative Neural Autoregressive Distribution Estimator (NADE-k)

Cited by in corpus (14)

• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Adversarial Feature Matching for Text Generation
• Adversarial Generation of Natural Language
• Z-Forcing: Training Stochastic Recurrent Networks
• Masked Non-Autoregressive Image Captioning
• eCommerceGAN : A Generative Adversarial Network for E-commerce
• Consensus-based Sequence Training for Video Captioning
• Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning
• DAL: Dual Adversarial Learning for Dialogue Generation
• Variational Bi-LSTMs
• Self-Supervised Dialogue Learning
• Bilingual-GAN: A Step Towards Parallel Text Generation
• Learning Powerful Policies by Using Consistent Dynamics Model
• Neural Sequence Model Training via $\alpha$-divergence Minimization