Massively Multilingual Neural Machine Translation

Abstract

Multilingual neural machine translation (NMT) enables training a single model that supports translation from multiple source languages into multiple target languages. In this paper, we push the limits of multilingual NMT in terms of number of languages being used. We perform extensive experiments in training massively multilingual NMT models, translating up to 102 languages to and from English within a single model. We explore different setups for training such models and analyze the trade-offs between translation quality and various modeling decisions. We report results on the publicly available TED talks multilingual corpus where we show that massively multilingual many-to-many models are effective in low resource settings, outperforming the previous state-of-the-art while supporting up to 59 languages. Our experiments on a large-scale dataset with 102 languages to and from English and up to one million examples per direction also show promising results, surpassing strong bilingual baselines and encouraging future work on massively multilingual NMT.

Accepted as a long paper in NAACL 2019

References in corpus (7)

• Sequence to Sequence Learning with Neural Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Achieving Human Parity on Automatic Chinese to English News Translation
• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
• Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
• Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
• Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation

Cited by in corpus (20)

• Language Models are Few-Shot Learners
• Multilingual Denoising Pre-training for Neural Machine Translation
• Natural Language Processing Advancements By Deep Learning: A Survey
• XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
• Time-aware Large Kernel Convolutions
• MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark
• Demonstrating Analog Inference on the BrainScaleS-2 Mobile System
• A Baseline Neural Machine Translation System for Indian Languages
• Fundamental Limits and Tradeoffs in Invariant Representation Learning
• Large-scale Pretraining for Neural Machine Translation with Tens of Billions of Sentence Pairs
• UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
• Robust Optimization for Multilingual Translation with Imbalanced Data
• Multitask Learning for Class-Imbalanced Discourse Classification
• A Survey on Low-Resource Neural Machine Translation
• Towards Reinforcement Learning for Pivot-based Neural Machine Translation with Non-autoregressive Transformer
• On the Language Coverage Bias for Neural Machine Translation
• Revisiting Modularized Multilingual NMT to Meet Industrial Demands
• Effects of Language Relatedness for Cross-lingual Transfer Learning in Character-Based Language Models
• Improving Target-side Lexical Transfer in Multilingual Neural Machine Translation
• A Dynamic Head Importance Computation Mechanism for Neural Machine Translation