A Survey on Low-Resource Neural Machine Translation

Abstract

Neural approaches have achieved state-of-the-art accuracy on machine translation but suffer from the high cost of collecting large scale parallel data. Thus, a lot of research has been conducted for neural machine translation (NMT) with very limited parallel data, i.e., the low-resource setting. In this paper, we provide a survey for low-resource NMT and classify related works into three categories according to the auxiliary data they used: (1) exploiting monolingual data of source and/or target languages, (2) exploiting data from auxiliary languages, and (3) exploiting multi-modal data. We hope that our survey can help researchers to better understand this field and inspire them to design better algorithms, and help industry practitioners to choose appropriate algorithms for their applications.

A short version has been submitted to IJCAI2021 Survey Track on Feb. 26th, 2021, accepted on Apr. 16th, 2021. 14 pages, 4 figures

References in corpus (23)

• Cross-lingual Language Model Pretraining
• Multilingual Denoising Pre-training for Neural Machine Translation
• MASS: Masked Sequence to Sequence Pre-training for Language Generation
• Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
• Word Translation Without Parallel Data
• CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
• Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation
• The Missing Ingredient in Zero-Shot Neural Machine Translation
• Bridging Neural Machine Translation and Bilingual Dictionaries
• Massively Multilingual Neural Machine Translation
• Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation
• Choosing Transfer Languages for Cross-Lingual Learning
• Unsupervised Neural Machine Translation with SMT as Posterior Regularization
• Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies
• From Words to Sentences: A Progressive Learning Approach for Zero-resource Machine Translation with Visual Pivots
• Bilingual Dictionary Based Neural Machine Translation without Using Parallel Sentences
• A Study of Multilingual Neural Machine Translation
• Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation
• Doubly-Attentive Decoder for Multi-modal Neural Machine Translation
• Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting
• Translating Translationese: A Two-Step Approach to Unsupervised Machine Translation
• Target Conditioned Sampling: Optimizing Data Selection for Multilingual Neural Machine Translation