Learning a Multi-Domain Curriculum for Neural Machine Translation

Abstract

Most data selection research in machine translation focuses on improving a single domain. We perform data selection for multiple domains at once. This is achieved by carefully introducing instance-level domain-relevance features and automatically constructing a training curriculum to gradually concentrate on multi-domain relevant and noise-reduced data batches. Both the choice of features and the use of curriculum are crucial for balancing and improving all domains, including out-of-domain. In large-scale experiments, the multi-domain curriculum simultaneously reaches or outperforms the individual performance and brings solid gains over no-curriculum training.

arXiv commentAccepted at ACL2020

References in corpus (10)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• Fast Domain Adaptation for Neural Machine Translation
• Curriculum Learning and Minibatch Bucketing in Neural Machine Translation
• An Empirical Exploration of Curriculum Learning for Neural Machine Translation
• Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translation
• Neural Machine Translation Training in a Multi-Domain Scenario
• Multi-Domain Neural Machine Translation
• Balancing Training for Multilingual Neural Machine Translation
• Optimizing Data Usage via Differentiable Rewards