Improving Bidirectional Decoding with Dynamic Target Semantics in Neural Machine Translation

Abstract

Generally, Neural Machine Translation models generate target words in a left-to-right (L2R) manner and fail to exploit any future (right) semantics information, which usually produces an unbalanced translation. Recent works attempt to utilize the right-to-left (R2L) decoder in bidirectional decoding to alleviate this problem. In this paper, we propose a novel \textbf{D}ynamic \textbf{I}nteraction \textbf{M}odule (\textbf{DIM}) to dynamically exploit target semantics from R2L translation for enhancing the L2R translation quality. Different from other bidirectional decoding approaches, DIM firstly extracts helpful target information through addressing and reading operations, then updates target semantics for tracking the interactive history. Additionally, we further introduce an \textbf{agreement regularization} term into the training objective to narrow the gap between L2R and R2L translations. Experimental results on NIST Chinese$\Rightarrow$English and WMT'16 English$\Rightarrow$Romanian translation tasks show that our system achieves significant improvements over baseline systems, which also reaches comparable results compared to the state-of-the-art Transformer model with much fewer parameters of it.

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Sequence to Sequence Learning with Neural Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Convolutional Sequence to Sequence Learning
• Achieving Human Parity on Automatic Chinese to English News Translation
• Asynchronous Bidirectional Decoding for Neural Machine Translation
• Agreement-based Joint Training for Bidirectional Attention-based Neural Machine Translation
• Neural Machine Translation with Key-Value Memory-Augmented Attention
• Synchronous Bidirectional Inference for Neural Sequence Generation
• Regularizing Neural Machine Translation by Target-bidirectional Agreement
• Towards Decoding as Continuous Optimization in Neural Machine Translation
• DTMT: A Novel Deep Transition Architecture for Neural Machine Translation

Cited by in corpus (1)

• Reciprocal Supervised Learning Improves Neural Machine Translation