Neural Sequence Model Training via $\alpha$-divergence Minimization

Abstract

We propose a new neural sequence model training method in which the objective function is defined by $\alpha$-divergence. We demonstrate that the objective function generalizes the maximum-likelihood (ML)-based and reinforcement learning (RL)-based objective functions as special cases (i.e., ML corresponds to $\alpha \to 0$ and RL to $\alpha \to 1$). We also show that the gradient of the objective function can be considered a mixture of ML- and RL-based objective gradients. The experimental results of a machine translation task show that minimizing the objective function with $\alpha >0$ outperforms $\alpha \to 0$, which corresponds to ML-based methods.

2017 ICML Workshop on Learning to Generate Natural Language (LGNL 2017)

References in corpus (11)

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• A Neural Conversational Model
• Effective Approaches to Attention-based Neural Machine Translation
• The Marginal Value of Adaptive Gradient Methods in Machine Learning
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• An Actor-Critic Algorithm for Sequence Prediction
• Adversarial Neural Machine Translation
• From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood