Unsupervised Neural Machine Translation Initialized by Unsupervised Statistical Machine Translation

Abstract

Recent work achieved remarkable results in training neural machine translation (NMT) systems in a fully unsupervised way, with new and dedicated architectures that rely on monolingual corpora only. In this work, we propose to define unsupervised NMT (UNMT) as NMT trained with the supervision of synthetic bilingual data. Our approach straightforwardly enables the use of state-of-the-art architectures proposed for supervised NMT by replacing human-made bilingual data with synthetic bilingual data for training. We propose to initialize the training of UNMT with synthetic bilingual data generated by unsupervised statistical machine translation (USMT). The UNMT system is then incrementally improved using back-translation. Our preliminary experiments show that our approach achieves a new state-of-the-art for unsupervised machine translation on the WMT16 German--English news translation task, for both translation directions.

arXiv commentpreliminary work

References in corpus (1)

• A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings

Cited by in corpus (8)

• An Effective Approach to Unsupervised Machine Translation
• A Call for More Rigor in Unsupervised Cross-lingual Learning
• Exploiting Out-of-Domain Parallel Data through Multilingual Transfer Learning for Low-Resource Neural Machine Translation
• A Survey on Low-Resource Neural Machine Translation
• Towards Unsupervised Grammatical Error Correction using Statistical Machine Translation with Synthetic Comparable Corpus
• Paraphrase Generation as Unsupervised Machine Translation
• Explicit Cross-lingual Pre-training for Unsupervised Machine Translation
• Do all Roads Lead to Rome? Understanding the Role of Initialization in Iterative Back-Translation