Neural Machine Translation with Key-Value Memory-Augmented Attention

Abstract

Although attention-based Neural Machine Translation (NMT) has achieved remarkable progress in recent years, it still suffers from issues of repeating and dropping translations. To alleviate these issues, we propose a novel key-value memory-augmented attention model for NMT, called KVMEMATT. Specifically, we maintain a timely updated keymemory to keep track of attention history and a fixed value-memory to store the representation of source sentence throughout the whole translation process. Via nontrivial transformations and iterative interactions between the two memories, the decoder focuses on more appropriate source word(s) for predicting the next target word at each decoding step, therefore can improve the adequacy of translations. Experimental results on Chinese=>English and WMT17 German<=>English translation tasks demonstrate the superiority of the proposed model.

arXiv commentAccepted at IJCAI 2018

References in corpus (7)

• Effective Approaches to Attention-based Neural Machine Translation
• Modeling Coverage for Neural Machine Translation
• Implicit Distortion and Fertility Models for Attention-based Encoder-Decoder NMT Model
• Coverage Embedding Models for Neural Machine Translation
• Search Engine Guided Non-Parametric Neural Machine Translation
• Memory-enhanced Decoder for Neural Machine Translation
• Memory-augmented Neural Machine Translation

Cited by in corpus (4)

• Learning Efficient Lexically-Constrained Neural Machine Translation with External Memory
• Improving Bidirectional Decoding with Dynamic Target Semantics in Neural Machine Translation
• DTMT: A Novel Deep Transition Architecture for Neural Machine Translation
• Dual Past and Future for Neural Machine Translation