Revisiting Low-Resource Neural Machine Translation: A Case Study

Abstract

It has been shown that the performance of neural machine translation (NMT) drops starkly in low-resource conditions, underperforming phrase-based statistical machine translation (PBSMT) and requiring large amounts of auxiliary data to achieve competitive results. In this paper, we re-assess the validity of these results, arguing that they are the result of lack of system adaptation to low-resource settings. We discuss some pitfalls to be aware of when training low-resource NMT systems, and recent techniques that have shown to be especially helpful in low-resource settings, resulting in a set of best practices for low-resource NMT. In our experiments on German--English with different amounts of IWSLT14 training data, we show that, without the use of any auxiliary monolingual or multilingual data, an optimized NMT system can outperform PBSMT with far less data than previously claimed. We also apply these techniques to a low-resource Korean-English dataset, surpassing previously reported results by 4 BLEU.

to appear at ACL 2019

References in corpus (8)

• Sequence to Sequence Learning with Neural Networks
• Dual Learning for Machine Translation
• Layer Normalization
• Unsupervised Statistical Machine Translation
• Pay Less Attention with Lightweight and Dynamic Convolutions
• A Teacher-Student Framework for Zero-Resource Neural Machine Translation
• Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction
• Neural machine translation for low-resource languages

Cited by in corpus (8)

• A Survey of Deep Learning Techniques for Neural Machine Translation
• Translating the Unseen? Yoruba-English MT in Low-Resource, Morphologically-Unmarked Settings
• Approaching Neural Chinese Word Segmentation as a Low-Resource Machine Translation Task
• Evaluating Low-Resource Machine Translation between Chinese and Vietnamese with Back-Translation
• Using Interlinear Glosses as Pivot in Low-Resource Multilingual Machine Translation
• Sicilian Translator: A Recipe for Low-Resource NMT
• Fully Synthetic Data Improves Neural Machine Translation with Knowledge Distillation
• Improving Zero-shot Multilingual Neural Machine Translation for Low-Resource Languages