Learning to Discriminate Noises for Incorporating External Information in Neural Machine Translation
arXiv:1810.10317
Abstract
Previous studies show that incorporating external information could improve the translation quality of Neural Machine Translation (NMT) systems. However, there are inevitably noises in the external information, severely reducing the benefit that the existing methods could receive from the incorporation. To tackle the problem, this study pays special attention to the discrimination of the noises during the incorporation. We argue that there exist two kinds of noise in this external information, i.e. global noise and local noise, which affect the translations for the whole sentence and for some specific words, respectively. Accordingly, we propose a general framework that learns to jointly discriminate both the global and local noises, so that the external information could be better leveraged. Our model is trained on the dataset derived from the original parallel corpus without any external labeled data or annotation. Experimental results in various real-world scenarios, language pairs, and neural architectures indicate that discriminating noises contributes to significant improvements in translation quality by being able to better incorporate the external information, even in very noisy conditions.
arXiv comment8 pages
References in corpus (19)
- Sequence to Sequence Learning with Neural Networks
- ADADELTA: An Adaptive Learning Rate Method
- Convolutional Sequence to Sequence Learning
- Effective Approaches to Attention-based Neural Machine Translation
- Incorporating Copying Mechanism in Sequence-to-Sequence Learning
- An Actor-Critic Algorithm for Sequence Prediction
- Modeling Coverage for Neural Machine Translation
- Pre-Translation for Neural Machine Translation
- Pointing the Unknown Words
- Incorporating Discrete Translation Lexicons into Neural Machine Translation
- Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search
- Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder
- Exploiting Cross-Sentence Context for Neural Machine Translation
- Memory-enhanced Decoder for Neural Machine Translation
- The University of Edinburgh's Neural MT Systems for WMT17
- Neural System Combination for Machine Translation
- Nematus: a Toolkit for Neural Machine Translation
- Neural Machine Translation with Word Predictions
- Translating Phrases in Neural Machine Translation