A Teacher-Student Framework for Zero-Resource Neural Machine Translation

Abstract

While end-to-end neural machine translation (NMT) has made remarkable progress recently, it still suffers from the data scarcity problem for low-resource language pairs and domains. In this paper, we propose a method for zero-resource NMT by assuming that parallel sentences have close probabilities of generating a sentence in a third language. Based on this assumption, our method is able to train a source-to-target NMT model ("student") without parallel corpora available, guided by an existing pivot-to-target NMT model ("teacher") on a source-pivot parallel corpus. Experimental results show that the proposed method significantly improves over a baseline pivot-based model by +3.0 BLEU points across various language pairs.

Accepted as a long paper by ACL 2017

References in corpus (10)

• Distilling the Knowledge in a Neural Network
• Sequence to Sequence Learning with Neural Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Do Deep Nets Really Need to be Deep?
• Sequence Level Training with Recurrent Neural Networks
• Dual Learning for Machine Translation
• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
• Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
• Zero-Resource Translation with Multi-Lingual Neural Machine Translation
• Neural Machine Translation with Pivot Languages

Cited by in corpus (42)

• Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
• Phrase-Based & Neural Unsupervised Machine Translation
• Unsupervised Machine Translation Using Monolingual Corpora Only
• The Missing Ingredient in Zero-Shot Neural Machine Translation
• Unsupervised Neural Machine Translation
• A Call for More Rigor in Unsupervised Cross-lingual Learning
• Zero-Resource Knowledge-Grounded Dialogue Generation
• Multilingual NMT with a language-independent attention bridge
• Corpus Augmentation by Sentence Segmentation for Low-Resource Neural Machine Translation
• Meta-Learning for Low-Resource Neural Machine Translation
• Revisiting Low-Resource Neural Machine Translation: A Case Study
• A neural interlingua for multilingual machine translation
• Towards Robust Neural Machine Translation
• Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies
• Triangular Architecture for Rare Language Translation
• Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations
• Contextual Parameter Generation for Universal Neural Machine Translation
• From Words to Sentences: A Progressive Learning Approach for Zero-resource Machine Translation with Visual Pivots
• Improving Back-Translation with Uncertainty-based Confidence Estimation
• Generalized Data Augmentation for Low-Resource Translation
• A Study of Multilingual Neural Machine Translation
• Language Graph Distillation for Low-Resource Machine Translation
• Mix and match networks: encoder-decoder alignment for zero-pair image translation
• Beyond English-Only Reading Comprehension: Experiments in Zero-Shot Multilingual Transfer for Bulgarian
• Zero-Resource Neural Machine Translation with Multi-Agent Communication Game
• Towards Neural Machine Translation with Partially Aligned Corpora
• Learning Representation Mapping for Relation Detection in Knowledge Base Question Answering
• Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting
• Contrastive Learning for Many-to-many Multilingual Neural Machine Translation
• A Stable and Effective Learning Strategy for Trainable Greedy Decoding
• A Brief Survey of Multilingual Neural Machine Translation
• Translating Translationese: A Two-Step Approach to Unsupervised Machine Translation
• A Survey on Low-Resource Neural Machine Translation
• Cross-lingual Supervision Improves Unsupervised Neural Machine Translation
• Unsupervised Pivot Translation for Distant Languages
• Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data
• Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
• A General Framework for Adaptation of Neural Machine Translation to Simultaneous Translation
• Self-Learning for Zero Shot Neural Machine Translation
• Autoregressive Knowledge Distillation through Imitation Learning
• Jointly Learning to Repair Code and Generate Commit Message
• Token-wise Curriculum Learning for Neural Machine Translation