A Comprehensive Survey of Multilingual Neural Machine Translation

Abstract

We present a survey on multilingual neural machine translation (MNMT), which has gained a lot of traction in the recent years. MNMT has been useful in improving translation quality as a result of translation knowledge transfer (transfer learning). MNMT is more promising and interesting than its statistical machine translation counterpart because end-to-end modeling and distributed representations open new avenues for research on machine translation. Many approaches have been proposed in order to exploit multilingual parallel corpora for improving translation quality. However, the lack of a comprehensive survey makes it difficult to determine which approaches are promising and hence deserve further exploration. In this paper, we present an in-depth survey of existing literature on MNMT. We first categorize various approaches based on their central use-case and then further categorize them based on resource scenarios, underlying modeling principles, core-issues and challenges. Wherever possible we address the strengths and weaknesses of several techniques by comparing them with each other. We also discuss the future directions that MNMT research might take. This paper is aimed towards both, beginners and experts in NMT. We hope this paper will serve as a starting point as well as a source of new ideas for researchers and engineers interested in MNMT.

This is an extended version of our survey paper on multilingual NMT. The previous version [arXiv:1905.05395] is rather condensed and is useful for speed-reading whereas this version is more beginner friendly. Under review at the computing surveys journal. We have intentionally decided to maintain both short and long versions of our survey paper for different reader groups

References in corpus (14)

• Distilling the Knowledge in a Neural Network
• Sequence to Sequence Learning with Neural Networks
• Neural Architecture Search with Reinforcement Learning
• Cross-lingual Language Model Pretraining
• Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
• SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
• Multilingual Neural Machine Translation with Knowledge Distillation
• Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
• The Missing Ingredient in Zero-Shot Neural Machine Translation
• Code-Switching for Enhancing NMT with Pre-Specified Translation
• Effective Strategies in Zero-Shot Neural Machine Translation
• Multilingual Multi-Domain Adaptation Approaches for Neural Machine Translation
• The ARIEL-CMU Systems for LoReHLT18

Cited by in corpus (3)

• Literature review on vulnerability detection using NLP technology
• Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters
• Language Tags Matter for Zero-Shot Neural Machine Translation