A Comprehensive Survey of Multilingual Neural Machine Translation

Abstract

We present a survey on multilingual neural machine translation (MNMT), which has gained a lot of traction in the recent years. MNMT has been useful in improving translation quality as a result of translation knowledge transfer (transfer learning). MNMT is more promising and interesting than its statistical machine translation counterpart because end-to-end modeling and distributed representations open new avenues for research on machine translation. Many approaches have been proposed in order to exploit multilingual parallel corpora for improving translation quality. However, the lack of a comprehensive survey makes it difficult to determine which approaches are promising and hence deserve further exploration. In this paper, we present an in-depth survey of existing literature on MNMT. We first categorize various approaches based on their central use-case and then further categorize them based on resource scenarios, underlying modeling principles, core-issues and challenges. Wherever possible we address the strengths and weaknesses of several techniques by comparing them with each other. We also discuss the future directions that MNMT research might take. This paper is aimed towards both, beginners and experts in NMT. We hope this paper will serve as a starting point as well as a source of new ideas for researchers and engineers interested in MNMT.

arXiv commentThis is an extended version of our survey paper on multilingual NMT. The previous version [arXiv:1905.05395] is rather condensed and is useful for speed-reading whereas this version is more beginner friendly. Under review at the computing surveys journal. We have intentionally decided to maintain both short and long versions of our survey paper for different reader groups

References in corpus (30)

• Distilling the Knowledge in a Neural Network
• Sequence to Sequence Learning with Neural Networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• An overview of gradient descent optimization algorithms
• Neural Architecture Search with Reinforcement Learning
• Cross-lingual Language Model Pretraining
• Exploiting Similarities among Languages for Machine Translation
• An Autoencoder Approach to Learning Bilingual Word Representations
• Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
• SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
• XNLI: Evaluating Cross-lingual Sentence Representations
• Multilingual Neural Machine Translation with Knowledge Distillation
• Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
• Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
• The Missing Ingredient in Zero-Shot Neural Machine Translation
• An Empirical Analysis of NMT-Derived Interlingual Embeddings and their Use in Parallel Sentence Identification
• Universal Neural Machine Translation for Extremely Low Resource Languages
• Code-Switching for Enhancing NMT with Pre-Specified Translation
• Effective Strategies in Zero-Shot Neural Machine Translation
• Improving Zero-Shot Translation of Low-Resource Languages
• Multilingual NMT with a language-independent attention bridge
• Adaptive Scheduling for Multi-Task Learning
• Enabling Multi-Source Neural Machine Translation By Concatenating Source Sentences In Multiple Languages
• Zero-Shot Dual Machine Translation
• Neural System Combination for Machine Translation
• Multi-Source Neural Machine Translation with Data Augmentation
• Multilingual Multi-Domain Adaptation Approaches for Neural Machine Translation
• MMCR4NLP: Multilingual Multiway Corpora Repository for Natural Language Processing
• The ARIEL-CMU Systems for LoReHLT18

Cited by in corpus (5)

• Evaluation of Text Generation: A Survey
• Literature review on vulnerability detection using NLP technology
• Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters
• Language Tags Matter for Zero-Shot Neural Machine Translation
• Generalising Multilingual Concept-to-Text NLG with Language Agnostic Delexicalisation