Neural Machine Translation Advised by Statistical Machine Translation

Abstract

Neural Machine Translation (NMT) is a new approach to machine translation that has made great progress in recent years. However, recent studies show that NMT generally produces fluent but inadequate translations (Tu et al. 2016b; Tu et al. 2016a; He et al. 2016; Tu et al. 2017). This is in contrast to conventional Statistical Machine Translation (SMT), which usually yields adequate but non-fluent translations. It is natural, therefore, to leverage the advantages of both models for better translations, and in this work we propose to incorporate SMT model into NMT framework. More specifically, at each decoding step, SMT offers additional recommendations of generated words based on the decoding information from NMT (e.g., the generated partial translation and attention history). Then we employ an auxiliary classifier to score the SMT recommendations and a gating function to combine the SMT recommendations with NMT generations, both of which are jointly trained within the NMT architecture in an end-to-end manner. Experimental results on Chinese-English translation show that the proposed approach achieves significant and consistent improvements over state-of-the-art NMT and SMT systems on multiple NIST test sets.

Accepted by AAAI 2017

References in corpus (11)

• ADADELTA: An Adaptive Learning Rate Method
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Incorporating Copying Mechanism in Sequence-to-Sequence Learning
• Modeling Coverage for Neural Machine Translation
• Syntactically Guided Neural Machine Translation
• Neural Machine Translation with Reconstruction
• Neural Machine Translation with External Phrase Memory
• Implicit Distortion and Fertility Models for Attention-based Encoder-Decoder NMT Model
• Incorporating Discrete Translation Lexicons into Neural Machine Translation
• A Deep Memory-based Architecture for Sequence-to-Sequence Learning
• Context Gates for Neural Machine Translation

Cited by in corpus (3)

• A GRU-Gated Attention Model for Neural Machine Translation
• Prior Knowledge Integration for Neural Machine Translation using Posterior Regularization
• Fusing Recency into Neural Machine Translation with an Inter-Sentence Gate Model