Triangular Architecture for Rare Language Translation

Abstract

Neural Machine Translation (NMT) performs poor on the low-resource language pair $(X,Z)$, especially when $Z$ is a rare language. By introducing another rich language $Y$, we propose a novel triangular training architecture (TA-NMT) to leverage bilingual data $(Y,Z)$ (may be small) and $(X,Y)$ (can be rich) to improve the translation performance of low-resource pairs. In this triangular architecture, $Z$ is taken as the intermediate latent variable, and translation models of $Z$ are jointly optimized with a unified bidirectional EM algorithm under the goal of maximizing the translation likelihood of $(X,Y)$. Empirical results demonstrate that our method significantly improves the translation quality of rare languages on MultiUN and IWSLT2012 datasets, and achieves even better performance combining back-translation methods.

arXiv commentAccepted to ACL 2018, 10 pages, 5 figures, 5 tables (with 5-5-5-5 high score)

References in corpus (8)

• Sequence to Sequence Learning with Neural Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• ADADELTA: An Adaptive Learning Rate Method
• Dual Learning for Machine Translation
• Sequence-Level Knowledge Distillation
• A Teacher-Student Framework for Zero-Resource Neural Machine Translation
• Minimum Risk Training for Neural Machine Translation
• Joint Training for Neural Machine Translation Models with Monolingual Data

Cited by in corpus (6)

• The Missing Ingredient in Zero-Shot Neural Machine Translation
• Zero-Resource Knowledge-Grounded Dialogue Generation
• A Variational Dirichlet Framework for Out-of-Distribution Detection
• A Survey on Low-Resource Neural Machine Translation
• Cross-lingual Supervision Improves Unsupervised Neural Machine Translation
• Rethinking Zero-shot Neural Machine Translation: From a Perspective of Latent Variables