Unsupervised Statistical Machine Translation

Abstract

While modern machine translation has relied on large parallel corpora, a recent line of work has managed to train Neural Machine Translation (NMT) systems from monolingual corpora only (Artetxe et al., 2018c; Lample et al., 2018). Despite the potential of this approach for low-resource settings, existing systems are far behind their supervised counterparts, limiting their practical interest. In this paper, we propose an alternative approach based on phrase-based Statistical Machine Translation (SMT) that significantly closes the gap with supervised systems. Our method profits from the modular architecture of SMT: we first induce a phrase table from monolingual corpora through cross-lingual embedding mappings, combine it with an n-gram language model, and fine-tune hyperparameters through an unsupervised MERT variant. In addition, iterative backtranslation improves results further, yielding, for instance, 14.08 and 26.22 BLEU points in WMT 2014 English-German and English-French, respectively, an improvement of more than 7-10 BLEU points over previous unsupervised systems, and closing the gap with supervised SMT (Moses trained on Europarl) down to 2-5 BLEU points. Our implementation is available at https://github.com/artetxem/monoses

EMNLP 2018

References in corpus (1)

• A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings

Cited by in corpus (59)

• Evaluating Word Embedding Models: Methods and Experimental Results
• Unsupervised Question Answering by Cloze Translation
• An Effective Approach to Unsupervised Machine Translation
• Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review
• Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
• Unsupervised Translation of Programming Languages
• A Call for More Rigor in Unsupervised Cross-lingual Learning
• Bilingual Lexicon Induction through Unsupervised Machine Translation
• A Study of Neural Matching Models for Cross-lingual IR
• When Does Unsupervised Machine Translation Work?
• A Survey of Orthographic Information in Machine Translation
• Break-It-Fix-It: Unsupervised Learning for Program Repair
• Revisiting Low-Resource Neural Machine Translation: A Case Study
• "A Passage to India": Pre-trained Word Embeddings for Indian Languages
• Low Resource Neural Machine Translation: A Benchmark for Five African Languages
• Exploration of Neural Machine Translation in Autoformalization of Mathematics in Mizar
• Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies
• Exploiting Out-of-Domain Parallel Data through Multilingual Transfer Learning for Low-Resource Neural Machine Translation
• Semi-Supervised Graph-to-Graph Translation
• Cross-model Back-translated Distillation for Unsupervised Machine Translation
• When and Why is Unsupervised Neural Machine Translation Useless?
• Neural Machine Translation: Challenges, Progress and Future
• Bilingual Dictionary Based Neural Machine Translation without Using Parallel Sentences
• Incorporating Word and Subword Units in Unsupervised Machine Translation Using Language Model Rescoring
• Monolingual and Parallel Corpora for Kangri Low Resource Language
• SJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task
• On the Effect of Word Order on Cross-lingual Sentiment Analysis
• R-VGAE: Relational-variational Graph Autoencoder for Unsupervised Prerequisite Chain Learning
• Lite Training Strategies for Portuguese-English and English-Portuguese Translation
• Cross-lingual Emotion Intensity Prediction
• Cross-lingual Supervision Improves Unsupervised Neural Machine Translation
• The TALP-UPC System for the WMT Similar Language Task: Statistical vs Neural Machine Translation
• Towards Unsupervised Grammatical Error Correction using Statistical Machine Translation with Synthetic Comparable Corpus
• Unsupervised Cross-Domain Prerequisite Chain Learning using Variational Graph Autoencoders
• Extremely low-resource machine translation for closely related languages
• Paraphrase Generation as Unsupervised Machine Translation
• CUNI Systems for the Unsupervised and Very Low Resource Translation Task in WMT20
• Towards Interlingua Neural Machine Translation
• Density Matching for Bilingual Word Embedding
• Duality Regularization for Unsupervised Bilingual Lexicon Induction
• Cross-lingual hate speech detection based on multilingual domain-specific word embeddings
• Explicit Cross-lingual Pre-training for Unsupervised Machine Translation
• The LMU Munich System for the WMT 2020 Unsupervised Machine Translation Shared Task
• Crosslingual Embeddings are Essential in UNMT for Distant Languages: An English to IndoAryan Case Study
• Unsupervised Neural Text Simplification
• Automatic Intent-Slot Induction for Dialogue Systems
• "Wikily" Supervised Neural Translation Tailored to Cross-Lingual Tasks
• Back-Training excels Self-Training at Unsupervised Domain Adaptation of Question Generation and Passage Retrieval
• Learning synchronous context-free grammars with multiple specialised non-terminals for hierarchical phrase-based translation
• Translation in the Wild
• From meaning to perception -- exploring the space between word and odor perception embeddings
• An Unsupervised Method for Building Sentence Simplification Corpora in Multiple Languages
• Learning to Pronounce Chinese Without a Pronunciation Dictionary
• HintedBT: Augmenting Back-Translation with Quality and Transliteration Hints
• Efficient Variational Graph Autoencoders for Unsupervised Cross-domain Prerequisite Chains
• Image Captioning with Visual Object Representations Grounded in the Textual Modality
• Understanding and Enhancing the Use of Context for Machine Translation
• Do all Roads Lead to Rome? Understanding the Role of Initialization in Iterative Back-Translation
• Scrambled Translation Problem: A Problem of Denoising UNMT