Rethinking Exposure Bias In Language Modeling
arXiv:1910.11235
Abstract
Exposure bias describes the phenomenon that a language model trained under the teacher forcing schema may perform poorly at the inference stage when its predictions are conditioned on its previous predictions unseen from the training corpus. Recently, several generative adversarial networks (GANs) and reinforcement learning (RL) methods have been introduced to alleviate this problem. Nonetheless, a common issue in RL and GANs training is the sparsity of reward signals. In this paper, we adopt two simple strategies, multi-range reinforcing, and multi-entropy sampling, to amplify and denoise the reward signal. Our model produces an improvement over competing models with regards to BLEU scores and road exam, a new metric we designed to measure the robustness against exposure bias in language models.
References in corpus (9)
- Playing Atari with Deep Reinforcement Learning
- Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
- An Actor-Critic Algorithm for Sequence Prediction
- How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
- Long Text Generation via Adversarial Training with Leaked Information
- Adversarial Ranking for Language Generation
- Connecting Generative Adversarial Networks and Actor-Critic Methods
- Evaluating Text GANs as Language Models
- Neural Program Synthesis By Self-Learning