Language-Independent Representor for Neural Machine Translation

Abstract

Current Neural Machine Translation (NMT) employs a language-specific encoder to represent the source sentence and adopts a language-specific decoder to generate target translation. This language-dependent design leads to large-scale network parameters and makes the duality of the parallel data underutilized. To address the problem, we propose in this paper a language-independent representor to replace the encoder and decoder by using weight sharing. This shared representor can not only reduce large portion of network parameters, but also facilitate us to fully explore the language duality by jointly training source-to-target, target-to-source, left-to-right and right-to-left translations within a multi-task learning framework. Experiments show that our proposed framework can obtain significant improvements over conventional NMT models on resource-rich and low-resource translation tasks with only a quarter of parameters.

References in corpus (12)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Convolutional Sequence to Sequence Learning
• Effective Approaches to Attention-based Neural Machine Translation
• Dual Learning for Machine Translation
• Achieving Human Parity on Automatic Chinese to English News Translation
• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
• Sequence-Level Knowledge Distillation
• Neural Machine Translation with Reconstruction
• Universal Neural Machine Translation for Extremely Low Resource Languages
• Incorporating Discrete Translation Lexicons into Neural Machine Translation
• Zero-Resource Translation with Multi-Lingual Neural Machine Translation
• Joint Training for Neural Machine Translation Models with Monolingual Data