Fusing Recency into Neural Machine Translation with an Inter-Sentence Gate Model

Abstract

Neural machine translation (NMT) systems are usually trained on a large amount of bilingual sentence pairs and translate one sentence at a time, ignoring inter-sentence information. This may make the translation of a sentence ambiguous or even inconsistent with the translations of neighboring sentences. In order to handle this issue, we propose an inter-sentence gate model that uses the same encoder to encode two adjacent sentences and controls the amount of information flowing from the preceding sentence to the translation of the current sentence with an inter-sentence gate. In this way, our proposed model can capture the connection between sentences and fuse recency from neighboring sentences into neural machine translation. On several NIST Chinese-English translation tasks, our experiments demonstrate that the proposed inter-sentence gate model achieves substantial improvements over the baseline.

arXiv commentAccepted by COLING2018.11 pages,4 figures. arXiv admin note: text overlap with arXiv:1711.11221

References in corpus (8)

• Sequence to Sequence Learning with Neural Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• ADADELTA: An Adaptive Learning Rate Method
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Effective Approaches to Attention-based Neural Machine Translation
• Memory-enhanced Decoder for Neural Machine Translation
• Context Gates for Neural Machine Translation
• Neural Machine Translation Advised by Statistical Machine Translation

Cited by in corpus (2)

• Towards Making the Most of Context in Neural Machine Translation
• Document-level Neural Machine Translation with Document Embeddings