Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
arXiv:1611.04558
Abstract
We propose a simple solution to use a single Neural Machine Translation (NMT) model to translate between multiple languages. Our solution requires no change in the model architecture from our base system but instead introduces an artificial token at the beginning of the input sentence to specify the required target language. The rest of the model, which includes encoder, decoder and attention, remains unchanged and is shared across all languages. Using a shared wordpiece vocabulary, our approach enables Multilingual NMT using a single model without any increase in parameters, which is significantly simpler than previous proposals for Multilingual NMT. Our method often improves the translation quality of all involved language pairs, even while keeping the total number of model parameters constant. On the WMT'14 benchmarks, a single multilingual model achieves comparable performance for EnglishFrench and surpasses state-of-the-art results for EnglishGerman. Similarly, a single multilingual model surpasses state-of-the-art results for FrenchEnglish and GermanEnglish on WMT'14 and WMT'15 benchmarks respectively. On production corpora, multilingual models of up to twelve language pairs allow for better translation of many individual pairs. In addition to improving the translation quality of language pairs that the model was trained with, our models can also learn to perform implicit bridging between language pairs never seen explicitly during training, showing that transfer learning and zero-shot translation is possible for neural translation. Finally, we show analyses that hints at a universal interlingua representation in our models and show some interesting examples when mixing languages.
References in corpus (2)
Cited by in corpus (55)
- One Model To Learn Them All
- Phrase-Based & Neural Unsupervised Machine Translation
- Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
- A Survey of Domain Adaptation for Neural Machine Translation
- XNLI: Evaluating Cross-lingual Sentence Representations
- Linking the Neural Machine Translation and the Prediction of Organic Chemistry Reactions
- A Teacher-Student Framework for Zero-Resource Neural Machine Translation
- The Missing Ingredient in Zero-Shot Neural Machine Translation
- Global-Locally Self-Attentive Dialogue State Tracker
- Generating Focussed Molecule Libraries for Drug Discovery with Recurrent Neural Networks
- Personalization in Goal-Oriented Dialog
- Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languages
- Machine Common Sense Concept Paper
- Universal Neural Machine Translation for Extremely Low Resource Languages
- Multilingual Neural Machine Translation With Soft Decoupled Encoding
- Multilingual NMT with a language-independent attention bridge
- Controlling the Output Length of Neural Machine Translation
- Towards Explainable and Controllable Open Domain Dialogue Generation with Dialogue Acts
- Semi-Supervised QA with Generative Domain-Adaptive Nets
- Regularization Learning Networks: Deep Learning for Tabular Datasets
- Adversarial Text Generation Without Reinforcement Learning
- Enabling Multi-Source Neural Machine Translation By Concatenating Source Sentences In Multiple Languages
- Building a Neural Machine Translation System Using Only Synthetic Parallel Data
- Memory-augmented Neural Machine Translation
- Unsupervised Cross-lingual Word Embedding by Multilingual Neural Language Models
- Using Artificial Tokens to Control Languages for Multilingual Image Caption Generation
- Domain Control for Neural Machine Translation
- Exploiting Linguistic Resources for Neural Machine Translation Using Multi-task Learning
- Trainable Greedy Decoding for Neural Machine Translation
- Unsupervised Neural Machine Translation with Weight Sharing
- Simple, Scalable Adaptation for Neural Machine Translation
- Transfer Learning for Neural Semantic Parsing
- Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Model
- Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction
- Deep Learning in Information Security
- Filtering and Mining Parallel Data in a Joint Multilingual Space
- Style Obfuscation by Invariance
- Controlling Text Complexity in Neural Machine Translation
- Continuous multilinguality with language vectors
- Learning to Control Latent Representations for Few-Shot Learning of Named Entities
- Lessons learned in multilingual grounded language learning
- MDInference: Balancing Inference Accuracy and Latency for Mobile Applications
- Towards a resource for multilingual lexicons: an MT assisted and human-in-the-loop multilingual parallel corpus with multi-word expression annotation
- Towards Interlingua Neural Machine Translation
- Dual Learning: Theoretical Study and an Algorithmic Extension
- Sicilian Translator: A Recipe for Low-Resource NMT
- Neural MultiVoice Models for Expressing Novel Personalities in Dialog
- Zero-shot Neural Transfer for Cross-lingual Entity Linking
- Simple Automatic Post-editing for Arabic-Japanese Machine Translation
- Massively Multilingual Neural Grapheme-to-Phoneme Conversion
- Neural Machine Translation for Multilingual Grapheme-to-Phoneme Conversion
- Incorporating Chinese Radicals Into Neural Machine Translation: Deeper Than Character Level
- The MeMAD Submission to the WMT18 Multimodal Translation Task
- Improving Word Representations: A Sub-sampled Unigram Distribution for Negative Sampling
- Cognate-aware morphological segmentation for multilingual neural translation