A Survey on Low-Resource Neural Machine Translation

Abstract

Neural approaches have achieved state-of-the-art accuracy on machine translation but suffer from the high cost of collecting large scale parallel data. Thus, a lot of research has been conducted for neural machine translation (NMT) with very limited parallel data, i.e., the low-resource setting. In this paper, we provide a survey for low-resource NMT and classify related works into three categories according to the auxiliary data they used: (1) exploiting monolingual data of source and/or target languages, (2) exploiting data from auxiliary languages, and (3) exploiting multi-modal data. We hope that our survey can help researchers to better understand this field and inspire them to design better algorithms, and help industry practitioners to choose appropriate algorithms for their applications.

arXiv commentA short version has been submitted to IJCAI2021 Survey Track on Feb. 26th, 2021, accepted on Apr. 16th, 2021. 14 pages, 4 figures

References in corpus (45)

• Cross-lingual Language Model Pretraining
• Exploring the Limits of Language Modeling
• Multilingual Denoising Pre-training for Neural Machine Translation
• A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
• MASS: Masked Sequence to Sequence Pre-training for Language Generation
• Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
• Word Translation Without Parallel Data
• CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
• A Survey of Domain Adaptation for Neural Machine Translation
• Investigating Backtranslation in Neural Machine Translation
• Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation
• Imagination improves Multimodal Translation
• Understanding Back-Translation at Scale
• A Teacher-Student Framework for Zero-Resource Neural Machine Translation
• The Missing Ingredient in Zero-Shot Neural Machine Translation
• Multilingual Neural Machine Translation with Task-Specific Attention
• Emergent Translation in Multi-Agent Communication
• Bridging Neural Machine Translation and Bilingual Dictionaries
• Universal Neural Machine Translation for Extremely Low Resource Languages
• Massively Multilingual Neural Machine Translation
• Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation
• Zero-Resource Translation with Multi-Lingual Neural Machine Translation
• Choosing Transfer Languages for Cross-Lingual Learning
• Unsupervised Neural Machine Translation Initialized by Unsupervised Statistical Machine Translation
• Meta-Learning for Low-Resource Neural Machine Translation
• A neural interlingua for multilingual machine translation
• Unsupervised Neural Machine Translation with SMT as Posterior Regularization
• Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies
• Triangular Architecture for Rare Language Translation
• A Universal Parent Model for Low-Resource Neural Machine Translation Transfer
• Simple, Scalable Adaptation for Neural Machine Translation
• Rapid Adaptation of Neural Machine Translation to New Languages
• Contextual Parameter Generation for Universal Neural Machine Translation
• Balancing Training for Multilingual Neural Machine Translation
• Bi-Directional Neural Machine Translation with Synthetic Parallel Data
• Bilingual Dictionary Based Neural Machine Translation without Using Parallel Sentences
• From Words to Sentences: A Progressive Learning Approach for Zero-resource Machine Translation with Visual Pivots
• A Study of Multilingual Neural Machine Translation
• Language Graph Distillation for Low-Resource Machine Translation
• Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation
• Doubly-Attentive Decoder for Multi-modal Neural Machine Translation
• Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting
• Translating Translationese: A Two-Step Approach to Unsupervised Machine Translation
• Target Conditioned Sampling: Optimizing Data Selection for Multilingual Neural Machine Translation