Memory-augmented Neural Machine Translation

Abstract

Neural machine translation (NMT) has achieved notable success in recent times, however it is also widely recognized that this approach has limitations with handling infrequent words and word pairs. This paper presents a novel memory-augmented NMT (M-NMT) architecture, which stores knowledge about how words (usually infrequently encountered ones) should be translated in a memory and then utilizes them to assist the neural model. We use this memory mechanism to combine the knowledge learned from a conventional statistical machine translation system and the rules learned by an NMT system, and also propose a solution for out-of-vocabulary (OOV) words based on this framework. Our experiments on two Chinese-English translation tasks demonstrated that the M-NMT architecture outperformed the NMT baseline by $9.0$ and $2.7$ BLEU points on the two tasks, respectively. Additionally, we found this architecture resulted in a much more effective OOV treatment compared to competitive methods.

References in corpus (9)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• ADADELTA: An Adaptive Learning Rate Method
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
• Addressing the Rare Word Problem in Neural Machine Translation
• Syntactically Guided Neural Machine Translation
• Coverage Embedding Models for Neural Machine Translation
• Incorporating Discrete Translation Lexicons into Neural Machine Translation
• Memory-enhanced Decoder for Neural Machine Translation

Cited by in corpus (7)

• A Survey of Deep Learning Techniques for Neural Machine Translation
• Deep Learning Based Chatbot Models
• Integrating Transformer and Paraphrase Rules for Sentence Simplification
• Neural Machine Translation with Key-Value Memory-Augmented Attention
• CM-Net: A Novel Collaborative Memory Network for Spoken Language Understanding
• Learning Efficient Lexically-Constrained Neural Machine Translation with External Memory
• Phrase Table as Recommendation Memory for Neural Machine Translation