Unsupervised Machine Translation Using Monolingual Corpora Only

Abstract

Machine translation has recently achieved impressive performance thanks to recent advances in deep learning and the availability of large-scale parallel corpora. There have been numerous attempts to extend these successes to low-resource language pairs, yet requiring tens of thousands of parallel sentences. In this work, we take this research direction to the extreme and investigate whether it is possible to learn to translate even without any parallel data. We propose a model that takes sentences from monolingual corpora in two different languages and maps them into the same latent space. By learning to reconstruct in both languages from this shared feature space, the model effectively learns to translate without using any labeled data. We demonstrate our model on two widely used datasets and two language pairs, reporting BLEU scores of 32.8 and 15.1 on the Multi30k and WMT English-French datasets, without using even a single parallel sentence at training time.

ICLR 2018

References in corpus (6)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Dual Learning for Machine Translation
• On Using Monolingual Corpora in Neural Machine Translation
• Unsupervised Cross-Domain Image Generation
• Word Translation Without Parallel Data
• Emergent Translation in Multi-Agent Communication

Cited by in corpus (74)

• Multilingual Denoising Pre-training for Neural Machine Translation
• Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
• Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
• Multimodal Generative Models for Scalable Weakly-Supervised Learning
• Unsupervised Question Answering by Cloze Translation
• Broad-UNet: Multi-scale feature learning for nowcasting tasks
• MeanSum: A Neural Model for Unsupervised Multi-document Abstractive Summarization
• Pre-training via Paraphrasing
• Multi-modal Sarcasm Detection and Humor Classification in Code-mixed Conversations
• An Online Multilingual Hate speech Recognition System
• Multiple-Attribute Text Style Transfer
• Zero-Resource Knowledge-Grounded Dialogue Generation
• Cross-lingual Retrieval for Iterative Self-Supervised Training
• Unsupervised Text Style Transfer using Language Models as Discriminators
• Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer
• Image Captioning with Very Scarce Supervised Data: Adversarial Semi-Supervised Learning Approach
• Evaluating prose style transfer with the Bible
• Educating Text Autoencoders: Latent Representation Guidance via Denoising
• Meta-Learning for Low-Resource Neural Machine Translation
• EmoGraph: Capturing Emotion Correlations using Graph Networks
• A Comprehensive Survey of Grammar Error Correction
• Unsupervised Cross-lingual Word Embedding by Multilingual Neural Language Models
• Unsupervised Neural Machine Translation with SMT as Posterior Regularization
• DOBF: A Deobfuscation Pre-Training Objective for Programming Languages
• Towards Zero-shot Cross-lingual Image Retrieval
• Unsupervised Pidgin Text Generation By Pivoting English Data and Self-Training
• Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
• Unsupervised Neural Machine Translation with Weight Sharing
• Rapid Adaptation of Neural Machine Translation to New Languages
• Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection
• TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising
• Towards Unsupervised Automatic Speech Recognition Trained by Unaligned Speech and Text only
• Simplify-then-Translate: Automatic Preprocessing for Black-Box Machine Translation
• Zero-Shot Fine-Grained Style Transfer: Leveraging Distributed Continuous Style Representations to Transfer To Unseen Styles
• Detecting Social Media Manipulation in Low-Resource Languages
• Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary
• Learning Unsupervised Word Mapping by Maximizing Mean Discrepancy
• PidginUNMT: Unsupervised Neural Machine Translation from West African Pidgin to English
• Incorporating Bilingual Dictionaries for Low Resource Semi-Supervised Neural Machine Translation
• Language Graph Distillation for Low-Resource Machine Translation
• Neural Machine Translation: A Review of Methods, Resources, and Tools
• Zero-Resource Cross-Domain Named Entity Recognition
• Just Ask:An Interactive Learning Framework for Vision and Language Navigation
• Unsupervised Dual Paraphrasing for Two-stage Semantic Parsing
• Learning to Read by Spelling: Towards Unsupervised Text Recognition
• Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing
• Translating the Unseen? Yoruba-English MT in Low-Resource, Morphologically-Unmarked Settings
• Learning Efficient Lexically-Constrained Neural Machine Translation with External Memory
• Aligning Vector-spaces with Noisy Supervised Lexicons
• DART: A Lightweight Quality-Suggestive Data-to-Text Annotation Tool
• BET: A Backtranslation Approach for Easy Data Augmentation in Transformer-based Paraphrase Identification Context
• Towards Reducing Bias in Gender Classification
• Neural Machine Translation: A Review and Survey
• NeuraCrypt: Hiding Private Health Data via Random Neural Networks for Public Training
• Towards Interlingua Neural Machine Translation
• Neural Data-to-Text Generation with LM-based Text Augmentation
• Worse WER, but Better BLEU? Leveraging Word Embedding as Intermediate in Multitask End-to-End Speech Translation
• Using Monolingual Data in Neural Machine Translation: a Systematic Study
• Paraphrase Generation as Unsupervised Machine Translation
• Explicit Cross-lingual Pre-training for Unsupervised Machine Translation
• Unsupervised Neural Dialect Translation with Commonality and Diversity Modeling
• Sentence transition matrix: An efficient approach that preserves sentence semantics
• Handshakes AI Research at CASE 2021 Task 1: Exploring different approaches for multilingual tasks
• Unsupervised Transfer Learning in Multilingual Neural Machine Translation with Cross-Lingual Word Embeddings
• Bilingual Dictionary-based Language Model Pretraining for Neural Machine Translation
• A Semi-Supervised Approach for Low-Resourced Text Generation
• Paraphrase Thought: Sentence Embedding Module Imitating Human Language Recognition
• Massive Styles Transfer with Limited Labeled Data
• Zero-Shot Language Transfer vs Iterative Back Translation for Unsupervised Machine Translation
• A Relaxed Matching Procedure for Unsupervised BLI
• Language Model-Driven Unsupervised Neural Machine Translation
• The test set for the TransCoder system
• Jointly Improving Language Understanding and Generation with Quality-Weighted Weak Supervision of Automatic Labeling
• From Caesar Cipher to Unsupervised Learning: A New Method for Classifier Parameter Estimation