Professor Forcing: A New Algorithm for Training Recurrent Networks

Abstract

The Teacher Forcing algorithm trains recurrent networks by supplying observed sequence values as inputs during training and using the network's own one-step-ahead predictions to do multi-step sampling. We introduce the Professor Forcing algorithm, which uses adversarial domain adaptation to encourage the dynamics of the recurrent network to be the same when training the network and when sampling from the network over multiple time steps. We apply Professor Forcing to language modeling, vocal synthesis on raw waveforms, handwriting generation, and image generation. Empirically we find that Professor Forcing acts as a regularizer, improving test likelihood on character level Penn Treebank and sequential MNIST. We also find that the model qualitatively improves samples, especially when sampling for a large number of time steps. This is supported by human evaluation of sample quality. Trade-offs between Professor Forcing and Scheduled Sampling are discussed. We produce T-SNEs showing that Professor Forcing successfully makes the dynamics of the network during training and sampling more similar.

NIPS 2016 Accepted Paper

References in corpus (13)

• Sequence to Sequence Learning with Neural Networks
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Attention-Based Models for Speech Recognition
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• Pixel Recurrent Neural Networks
• DRAW: A Recurrent Neural Network For Image Generation
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
• MADE: Masked Autoencoder for Distribution Estimation
• Domain-Adversarial Neural Networks
• An Actor-Critic Algorithm for Sequence Prediction
• How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
• Iterative Neural Autoregressive Distribution Estimator (NADE-k)

Cited by in corpus (124)

• Deep Reinforcement Learning: An Overview
• Deep Learning for Time Series Forecasting: Tutorial and Literature Survey
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• A Multi-Horizon Quantile Recurrent Forecaster
• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• Adversarial Learning for Neural Dialogue Generation
• A Survey on Neural Speech Synthesis
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Adversarial Feature Matching for Text Generation
• Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP
• Deep learning algorithm for data-driven simulation of noisy dynamical system
• Adversarial Generation of Natural Language
• Listen and Fill in the Missing Letters: Non-Autoregressive Transformer for Speech Recognition
• Counterpoint by Convolution
• Machine Learning for Spatiotemporal Sequence Forecasting: A Survey
• Language-guided Navigation via Cross-Modal Grounding and Alternate Adversarial Learning
• Adversarial Neural Machine Translation
• Dance Revolution: Long-Term Dance Generation with Music via Curriculum Learning
• ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis
• Unsupervised Text Style Transfer using Language Models as Discriminators
• Short term prediction of demand for ride hailing services: A deep learning approach
• Seq2Slate: Re-ranking and Slate Optimization with RNNs
• A Semantics-Assisted Video Captioning Model Trained with Scheduled Sampling
• Motion Generation Using Bilateral Control-Based Imitation Learning with Autoregressive Learning
• A Tutorial on Deep Latent Variable Models of Natural Language
• Z-Forcing: Training Stochastic Recurrent Networks
• Show, Adapt and Tell: Adversarial Training of Cross-domain Image Captioner
• Regularizing RNNs for Caption Generation by Reconstructing The Past with The Present
• MPC-Inspired Neural Network Policies for Sequential Decision Making
• Masked Non-Autoregressive Image Captioning
• CAA-Net: Conditional Atrous CNNs with Attention for Explainable Device-robust Acoustic Scene Classification
• Music SketchNet: Controllable Music Generation via Factorized Representations of Pitch and Rhythm
• Improving Sequence-to-Sequence Learning via Optimal Transport
• Neural Language Generation: Formulation, Methods, and Evaluation
• Adversarial Text Generation Without Reinforcement Learning
• Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation
• Parallel Tacotron: Non-Autoregressive and Controllable TTS
• CoT: Cooperative Training for Generative Modeling of Discrete Data
• Review of end-to-end speech synthesis technology based on deep learning
• eCommerceGAN : A Generative Adversarial Network for E-commerce
• Towards Robust Neural Machine Translation
• Consensus-based Sequence Training for Video Captioning
• Fast Transient Simulation of High-Speed Channels Using Recurrent Neural Network
• Learning Dynamics Model in Reinforcement Learning by Incorporating the Long Term Future
• Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA
• Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning
• SALSA-TEXT : self attentive latent space based adversarial text generation
• Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
• Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation
• Human Motion Prediction via Learning Local Structure Representations and Temporal Dependencies
• Temporal Difference Variational Auto-Encoder
• CDL: Curriculum Dual Learning for Emotion-Controllable Response Generation
• Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets
• DAL: Dual Adversarial Learning for Dialogue Generation
• A Comprehensive Evaluation of Multi-task Learning and Multi-task Pre-training on EHR Time-series Data
• Discriminative Adversarial Search for Abstractive Summarization
• Towards Diverse Paraphrase Generation Using Multi-Class Wasserstein GAN
• Multivariate, Multistep Forecasting, Reconstruction and Feature Selection of Ocean Waves via Recurrent and Sequence-to-Sequence Networks
• KuroNet: Pre-Modern Japanese Kuzushiji Character Recognition with Deep Learning
• Topic-Preserving Synthetic News Generation: An Adversarial Deep Reinforcement Learning Approach
• PETGEN: Personalized Text Generation Attack on Deep Sequence Embedding-based Classification Models
• Point-less: More Abstractive Summarization with Pointer-Generator Networks
• Multi-Modal Generative Adversarial Network for Short Product Title Generation in Mobile E-Commerce
• Variational Bi-LSTMs
• What Makes A Good Story? Designing Composite Rewards for Visual Storytelling
• BERT as a Teacher: Contextual Embeddings for Sequence-Level Reward
• A New GAN-based End-to-End TTS Training Algorithm
• Self-Supervised Dialogue Learning
• Bilingual-GAN: A Step Towards Parallel Text Generation
• A Novel Fusion of Attention and Sequence to Sequence Autoencoders to Predict Sleepiness From Speech
• Reflective Decoding Network for Image Captioning
• Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation
• Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation
• Rescuing neural spike train models from bad MLE
• Generating Music with a Self-Correcting Non-Chronological Autoregressive Model
• Predictive Modeling: BIM Command Recommendation Based on Large-scale Usage Logs
• DLGNet: A Transformer-based Model for Dialogue Response Generation
• Attention Forcing for Sequence-to-sequence Model Training
• Improved Autoregressive Modeling with Distribution Smoothing
• SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environments
• MALCOM: Generating Malicious Comments to Attack Neural Fake News Detection Models
• Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognition
• From Credit Assignment to Entropy Regularization: Two New Algorithms for Neural Sequence Prediction
• Temporal LiDAR Frame Prediction for Autonomous Driving
• On the Discrepancy between Density Estimation and Sequence Generation
• Deep Reinforcement Learning with Distributional Semantic Rewards for Abstractive Summarization
• Learning Powerful Policies by Using Consistent Dynamics Model
• Alleviate Exposure Bias in Sequence Prediction \\ with Recurrent Neural Networks
• Attention Forcing for Machine Translation
• Sequence-Level Training for Non-Autoregressive Neural Machine Translation
• Adversarial Bootstrapping for Dialogue Model Training
• Video captioning with stacked attention and semantic hard pull
• Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation
• Using Monolingual Data in Neural Machine Translation: a Systematic Study
• Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification
• Improving Text Generation with Student-Forcing Optimal Transport
• Adaptive Correlated Monte Carlo for Contextual Categorical Sequence Generation
• Neural Machine Translation: A Review and Survey
• Variational Dynamic Mixtures
• Transferable Natural Language Interface to Structured Queries aided by Adversarial Generation
• Understanding attention-based encoder-decoder networks: a case study with chess scoresheet recognition
• AgreeSum: Agreement-Oriented Multi-Document Summarization
• Proving Equivalence Between Complex Expressions Using Graph-to-Sequence Neural Models
• To Beam Or Not To Beam: That is a Question of Cooperation for Language GANs
• Policy Gradients Incorporating the Future
• Forecasting Black Sigatoka Infection Risks with Latent Neural ODEs
• Adversarial Decomposition of Text Representation
• Deep Transfer Learning for Thermal Dynamics Modeling in Smart Buildings
• Deep Structured Reactive Planning
• Learning to Transfer Visual Effects from Videos to Images
• Generating Self-Contained and Summary-Centric Question Answer Pairs via Differentiable Reward Imitation Learning
• Hybrid Physics and Deep Learning Model for Interpretable Vehicle State Prediction
• Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation?
• Towards conceptual generalization in the embedding space
• Policy Message Passing: A New Algorithm for Probabilistic Graph Inference
• Hidden State Guidance: Improving Image Captioning using An Image Conditioned Autoencoder
• Predicting the Ordering of Characters in Japanese Historical Documents
• Positioning yourself in the maze of Neural Text Generation: A Task-Agnostic Survey
• Semi-Implicit Stochastic Recurrent Neural Networks
• Spatio-Temporal Event Segmentation and Localization for Wildlife Extended Videos
• Actions Generation from Captions
• Teaching Autoregressive Language Models Complex Tasks By Demonstration
• Adversarial Machine Learning in Text Analysis and Generation
• Neural Sequence Model Training via $\alpha$-divergence Minimization