An Effective Approach to Unsupervised Machine Translation

Abstract

While machine translation has traditionally relied on large amounts of parallel corpora, a recent research line has managed to train both Neural Machine Translation (NMT) and Statistical Machine Translation (SMT) systems using monolingual corpora only. In this paper, we identify and address several deficiencies of existing unsupervised SMT approaches by exploiting subword information, developing a theoretically well founded unsupervised tuning method, and incorporating a joint refinement procedure. Moreover, we use our improved SMT system to initialize a dual NMT model, which is further fine-tuned through on-the-fly back-translation. Together, we obtain large improvements over the previous state-of-the-art in unsupervised machine translation. For instance, we get 22.5 BLEU points in English-to-German WMT 2014, 5.5 points more than the previous best unsupervised system, and 0.5 points more than the (supervised) shared task winner back in 2014.

ACL 2019

References in corpus (6)

• Cross-lingual Language Model Pretraining
• Dual Learning for Machine Translation
• A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
• Unsupervised Statistical Machine Translation
• Unsupervised Neural Machine Translation Initialized by Unsupervised Statistical Machine Translation
• Unsupervised Neural Machine Translation with SMT as Posterior Regularization

Cited by in corpus (13)

• A Probabilistic Formulation of Unsupervised Text Style Transfer
• A Call for More Rigor in Unsupervised Cross-lingual Learning
• Bilingual Lexicon Induction through Unsupervised Machine Translation
• When Does Unsupervised Machine Translation Work?
• Cross-model Back-translated Distillation for Unsupervised Machine Translation
• Neural Machine Translation: A Review of Methods, Resources, and Tools
• SeqGenSQL -- A Robust Sequence Generation Model for Structured Query Language
• Cross-lingual Supervision Improves Unsupervised Neural Machine Translation
• Rapid Domain Adaptation for Machine Translation with Monolingual Data
• Paraphrase Generation as Unsupervised Machine Translation
• Do all Roads Lead to Rome? Understanding the Role of Initialization in Iterative Back-Translation
• Data Augmentation with Unsupervised Machine Translation Improves the Structural Similarity of Cross-lingual Word Embeddings
• Scrambled Translation Problem: A Problem of Denoising UNMT