Unsupervised Machine Translation Using Monolingual Corpora Only

Abstract

Machine translation has recently achieved impressive performance thanks to recent advances in deep learning and the availability of large-scale parallel corpora. There have been numerous attempts to extend these successes to low-resource language pairs, yet requiring tens of thousands of parallel sentences. In this work, we take this research direction to the extreme and investigate whether it is possible to learn to translate even without any parallel data. We propose a model that takes sentences from monolingual corpora in two different languages and maps them into the same latent space. By learning to reconstruct in both languages from this shared feature space, the model effectively learns to translate without using any labeled data. We demonstrate our model on two widely used datasets and two language pairs, reporting BLEU scores of 32.8 and 15.1 on the Multi30k and WMT English-French datasets, without using even a single parallel sentence at training time.

arXiv commentICLR 2018

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Style Transfer from Non-Parallel Text by Cross-Alignment
• Dual Learning for Machine Translation
• On Using Monolingual Corpora in Neural Machine Translation
• Unsupervised Image-to-Image Translation Networks
• Unsupervised Cross-Domain Image Generation
• Word Translation Without Parallel Data
• Toward Controlled Generation of Text
• A Teacher-Student Framework for Zero-Resource Neural Machine Translation
• Controllable Invariance through Adversarial Feature Learning
• Emergent Translation in Multi-Agent Communication
• Zero-Resource Translation with Multi-Lingual Neural Machine Translation

Cited by in corpus (75)

• Multilingual Denoising Pre-training for Neural Machine Translation
• Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
• Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
• A Survey Of Cross-lingual Word Embedding Models
• Multimodal Generative Models for Scalable Weakly-Supervised Learning
• Unsupervised Question Answering by Cloze Translation
• Broad-UNet: Multi-scale feature learning for nowcasting tasks
• MeanSum: A Neural Model for Unsupervised Multi-document Abstractive Summarization
• Pre-training via Paraphrasing
• Multi-modal Sarcasm Detection and Humor Classification in Code-mixed Conversations
• An Online Multilingual Hate speech Recognition System
• Multiple-Attribute Text Style Transfer
• Zero-Resource Knowledge-Grounded Dialogue Generation
• Cross-lingual Retrieval for Iterative Self-Supervised Training
• Unsupervised Text Style Transfer using Language Models as Discriminators
• Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer
• Evaluating prose style transfer with the Bible
• Image Captioning with Very Scarce Supervised Data: Adversarial Semi-Supervised Learning Approach
• Educating Text Autoencoders: Latent Representation Guidance via Denoising
• Meta-Learning for Low-Resource Neural Machine Translation
• EmoGraph: Capturing Emotion Correlations using Graph Networks
• A Comprehensive Survey of Grammar Error Correction
• Unsupervised Neural Machine Translation with SMT as Posterior Regularization
• Unsupervised Cross-lingual Word Embedding by Multilingual Neural Language Models
• DOBF: A Deobfuscation Pre-Training Objective for Programming Languages
• Unsupervised Pidgin Text Generation By Pivoting English Data and Self-Training
• Towards Zero-shot Cross-lingual Image Retrieval
• Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
• Unsupervised Neural Machine Translation with Weight Sharing
• Rapid Adaptation of Neural Machine Translation to New Languages
• TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising
• Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection
• Simplify-then-Translate: Automatic Preprocessing for Black-Box Machine Translation
• Towards Unsupervised Automatic Speech Recognition Trained by Unaligned Speech and Text only
• Zero-Shot Fine-Grained Style Transfer: Leveraging Distributed Continuous Style Representations to Transfer To Unseen Styles
• Detecting Social Media Manipulation in Low-Resource Languages
• Learning Unsupervised Word Mapping by Maximizing Mean Discrepancy
• Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary
• Language Graph Distillation for Low-Resource Machine Translation
• Neural Machine Translation: A Review of Methods, Resources, and Tools
• PidginUNMT: Unsupervised Neural Machine Translation from West African Pidgin to English
• Incorporating Bilingual Dictionaries for Low Resource Semi-Supervised Neural Machine Translation
• Zero-Resource Cross-Domain Named Entity Recognition
• Just Ask:An Interactive Learning Framework for Vision and Language Navigation
• Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing
• Translating the Unseen? Yoruba-English MT in Low-Resource, Morphologically-Unmarked Settings
• Learning to Read by Spelling: Towards Unsupervised Text Recognition
• Learning Efficient Lexically-Constrained Neural Machine Translation with External Memory
• Unsupervised Dual Paraphrasing for Two-stage Semantic Parsing
• DART: A Lightweight Quality-Suggestive Data-to-Text Annotation Tool
• BET: A Backtranslation Approach for Easy Data Augmentation in Transformer-based Paraphrase Identification Context
• Towards Reducing Bias in Gender Classification
• Aligning Vector-spaces with Noisy Supervised Lexicons
• Neural Data-to-Text Generation with LM-based Text Augmentation
• Towards Interlingua Neural Machine Translation
• Worse WER, but Better BLEU? Leveraging Word Embedding as Intermediate in Multitask End-to-End Speech Translation
• Paraphrase Generation as Unsupervised Machine Translation
• Neural Machine Translation: A Review and Survey
• Using Monolingual Data in Neural Machine Translation: a Systematic Study
• NeuraCrypt: Hiding Private Health Data via Random Neural Networks for Public Training
• Unsupervised Transfer Learning in Multilingual Neural Machine Translation with Cross-Lingual Word Embeddings
• A Semi-Supervised Approach for Low-Resourced Text Generation
• Handshakes AI Research at CASE 2021 Task 1: Exploring different approaches for multilingual tasks
• Bilingual Dictionary-based Language Model Pretraining for Neural Machine Translation
• Unsupervised Neural Dialect Translation with Commonality and Diversity Modeling
• Sentence transition matrix: An efficient approach that preserves sentence semantics
• Explicit Cross-lingual Pre-training for Unsupervised Machine Translation
• The test set for the TransCoder system
• Jointly Improving Language Understanding and Generation with Quality-Weighted Weak Supervision of Automatic Labeling
• A Relaxed Matching Procedure for Unsupervised BLI
• Language Model-Driven Unsupervised Neural Machine Translation
• Zero-Shot Language Transfer vs Iterative Back Translation for Unsupervised Machine Translation
• Massive Styles Transfer with Limited Labeled Data
• From Caesar Cipher to Unsupervised Learning: A New Method for Classifier Parameter Estimation
• Paraphrase Thought: Sentence Embedding Module Imitating Human Language Recognition