Bi-Directional Neural Machine Translation with Synthetic Parallel Data

Abstract

Despite impressive progress in high-resource settings, Neural Machine Translation (NMT) still struggles in low-resource and out-of-domain scenarios, often failing to match the quality of phrase-based translation. We propose a novel technique that combines back-translation and multilingual NMT to improve performance in these difficult cases. Our technique trains a single model for both directions of a language pair, allowing us to back-translate source or target monolingual data without requiring an auxiliary model. We then continue training on the augmented parallel data, enabling a cycle of improvement for a single model that can incorporate any source, target, or parallel data to improve both translation directions. As a byproduct, these models can reduce training and deployment costs significantly compared to uni-directional models. Extensive experiments show that our technique outperforms standard back-translation in low-resource scenarios, improves quality on cross-domain tasks, and effectively reduces costs across the board.

arXiv commentAccepted at the 2nd Workshop on Neural Machine Translation and Generation (WNMT 2018)

References in corpus (5)

• Sequence to Sequence Learning with Neural Networks
• Dual Learning for Machine Translation
• Layer Normalization
• Sockeye: A Toolkit for Neural Machine Translation
• Joint Training for Neural Machine Translation Models with Monolingual Data

Cited by in corpus (5)

• Understanding Back-Translation at Scale
• Multi-Task Neural Models for Translating Between Styles Within and Across Languages
• A Survey on Low-Resource Neural Machine Translation
• Findings of the Second Workshop on Neural Machine Translation and Generation
• Iterative Batch Back-Translation for Neural Machine Translation: A Conceptual Model