Addressing the Rare Word Problem in Neural Machine Translation

Abstract

Neural Machine Translation (NMT) is a new approach to machine translation that has shown promising results that are comparable to traditional approaches. A significant weakness in conventional NMT systems is their inability to correctly translate very rare words: end-to-end NMTs tend to have relatively small vocabularies with a single unk symbol that represents every possible out-of-vocabulary (OOV) word. In this paper, we propose and implement an effective technique to address this problem. We train an NMT system on data that is augmented by the output of a word alignment algorithm, allowing the NMT system to emit, for each OOV word in the target sentence, the position of its corresponding word in the source sentence. This information is later utilized in a post-processing step that translates every OOV word using a dictionary. Our experiments on the WMT14 English to French translation task show that this method provides a substantial improvement of up to 2.8 BLEU points over an equivalent NMT system that does not use this technique. With 37.5 BLEU points, our NMT system is the first to surpass the best result achieved on a WMT14 contest task.

ACL 2015 camera-ready version

References in corpus (1)

• Recurrent Neural Network Regularization

Cited by in corpus (33)

• A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
• On Using Monolingual Corpora in Neural Machine Translation
• Grammar as a Foreign Language
• A Hierarchical Neural Autoencoder for Paragraphs and Documents
• A Survey of Deep Learning Techniques for Neural Machine Translation
• Efficient Summarization with Read-Again and Copy Mechanism
• Reading Scene Text with Attention Convolutional Sequence Modeling
• Automated Software Vulnerability Assessment with Concept Drift
• Code-Switching for Enhancing NMT with Pre-Specified Translation
• Embedding Word Similarity with Neural Machine Translation
• Table-to-text Generation by Structure-aware Seq2seq Learning
• Neural Text Generation: A Practical Guide
• Temporal Learning and Sequence Modeling for a Job Recommender System
• Comparison of Deep learning models on time series forecasting : a case study of Dissolved Oxygen Prediction
• Hindi to English: Transformer-Based Neural Machine Translation
• Memory-augmented Neural Machine Translation
• Differentiable Top-k Operator with Optimal Transport
• Neural Machine Translation with Recurrent Attention Modeling
• Neural Contextual Conversation Learning with Labeled Question-Answering Pairs
• SMT vs NMT: A Comparison over Hindi & Bengali Simple Sentences
• Exploring the limits of Concurrency in ML Training on Google TPUs
• Log-Linear RNNs: Towards Recurrent Neural Networks with Flexible Prior Knowledge
• Machine Translation : From Statistical to modern Deep-learning practices
• Deep execution monitor for robot assistive tasks
• Teaching Machines to Converse
• An Efficient Character-Level Neural Machine Translation
• A Convolutional Neural Network for Language-Agnostic Source Code Summarization
• Context Models for OOV Word Translation in Low-Resource Languages
• Sequence-to-Sequence Modeling for Action Identification at High Temporal Resolution
• Urdu-English Machine Transliteration using Neural Networks
• NEJM-enzh: A Parallel Corpus for English-Chinese Translation in the Biomedical Domain
• A Review of Dialogue Systems: From Trained Monkeys to Stochastic Parrots
• Translation, Sentiment and Voices: A Computational Model to Translate and Analyse Voices from Real-Time Video Calling