Learning to Remember Translation History with a Continuous Cache
arXiv:1711.09367
Abstract
Existing neural machine translation (NMT) models generally translate sentences in isolation, missing the opportunity to take advantage of document-level information. In this work, we propose to augment NMT models with a very light-weight cache-like memory network, which stores recent hidden representations as translation history. The probability distribution over generated words is updated online depending on the translation history retrieved from the memory, endowing NMT models with the capability to dynamically adapt over time. Experiments on multiple domains with different topics and styles show the effectiveness of the proposed approach with negligible impact on the computational cost.
Accepted by TACL 2018
References in corpus (17)
- Improving neural networks by preventing co-adaptation of feature detectors
- ADADELTA: An Adaptive Learning Rate Method
- A Neural Conversational Model
- Effective Approaches to Attention-based Neural Machine Translation
- A Convolutional Neural Network for Modelling Sentences
- Incorporating Copying Mechanism in Sequence-to-Sequence Learning
- Learning to Remember Rare Events
- Modeling Coverage for Neural Machine Translation
- Document Context Language Models
- Pointing the Unknown Words
- Neural Machine Translation with Reconstruction
- Exploiting Cross-Sentence Context for Neural Machine Translation
- Learning to Create and Reuse Words in Open-Vocabulary Neural Language Modeling
- Modeling Source Syntax for Neural Machine Translation
- Recurrent Memory Networks for Language Modeling
- Translating Phrases in Neural Machine Translation
- Chunk-Based Bi-Scale Decoder for Neural Machine Translation