Cross-lingual Supervision Improves Unsupervised Neural Machine Translation
arXiv:2004.03137
Abstract
Neural machine translation~(NMT) is ineffective for zero-resource languages. Recent works exploring the possibility of unsupervised neural machine translation (UNMT) with only monolingual data can achieve promising results. However, there are still big gaps between UNMT and NMT with parallel supervision. In this work, we introduce a multilingual unsupervised NMT (\method) framework to leverage weakly supervised signals from high-resource language pairs to zero-resource translation directions. More specifically, for unsupervised language pairs \texttt{En-De}, we can make full use of the information from parallel dataset \texttt{En-Fr} to jointly train the unsupervised translation directions all in one model. \method is based on multilingual models which require no changes to the standard unsupervised NMT. Empirical results demonstrate that \method significantly improves the translation quality by more than 3 BLEU score on six benchmark unsupervised translation directions.
arXiv commentNAACL Industry Track
References in corpus (9)
- Convolutional Sequence to Sequence Learning
- Multilingual Denoising Pre-training for Neural Machine Translation
- Dual Learning for Machine Translation
- MASS: Masked Sequence to Sequence Pre-training for Language Generation
- Unsupervised Statistical Machine Translation
- An Effective Approach to Unsupervised Machine Translation
- A Teacher-Student Framework for Zero-Resource Neural Machine Translation
- Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information
- Triangular Architecture for Rare Language Translation