Incorporating Discrete Translation Lexicons into Neural Machine Translation

Abstract

Neural machine translation (NMT) often makes mistakes in translating low-frequency content words that are essential to understanding the meaning of the sentence. We propose a method to alleviate this problem by augmenting NMT systems with discrete translation lexicons that efficiently encode translations of these low-frequency words. We describe a method to calculate the lexicon probability of the next word in the translation candidate by using the attention vector of the NMT model to select which source word lexical probabilities the model should focus on. We test two methods to combine this probability with the standard NMT probability: (1) using it as a bias, and (2) linear interpolation. Experiments on two corpora show an improvement of 2.0-2.3 BLEU and 0.13-0.44 NIST score, and faster convergence time.

Accepted at EMNLP 2016

References in corpus (2)

• Sequence to Sequence Learning with Neural Networks
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches

Cited by in corpus (15)

• A Survey of Domain Adaptation for Neural Machine Translation
• Pre-Translation for Neural Machine Translation
• ShapeWorld - A new test methodology for multimodal language understanding
• Neural Machine Translation with Supervised Attention
• Memory-augmented Neural Machine Translation
• Prior Knowledge Integration for Neural Machine Translation using Posterior Regularization
• Translating Phrases in Neural Machine Translation
• Improving Neural Machine Translation through Phrase-based Forced Decoding
• Neural Machine Translation Advised by Statistical Machine Translation
• ReWE: Regressing Word Embeddings for Regularization of Neural Machine Translation Systems
• Learning to Discriminate Noises for Incorporating External Information in Neural Machine Translation
• Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings
• Phrase Table as Recommendation Memory for Neural Machine Translation
• Language-Independent Representor for Neural Machine Translation
• Massively Parallel Cross-Lingual Learning in Low-Resource Target Language Translation