Neural Machine Translation Training in a Multi-Domain Scenario

Abstract

In this paper, we explore alternative ways to train a neural machine translation system in a multi-domain scenario. We investigate data concatenation (with fine tuning), model stacking (multi-level fine tuning), data selection and multi-model ensemble. Our findings show that the best translation quality can be achieved by building an initial system on a concatenation of available out-of-domain data and then fine-tuning it on in-domain data. Model stacking works best when training begins with the furthest out-of-domain data and the model is incrementally fine-tuned with the next furthest domain and so on. Data selection did not give the best results, but can be considered as a decent compromise between training time and translation quality. A weighted ensemble of different individual models performed better than data selection. It is beneficial in a scenario when there is no time for fine-tuning an already trained model.

arXiv comment8 pages

References in corpus (5)

• Fast Domain Adaptation for Neural Machine Translation
• An Empirical Comparison of Simple Domain Adaptation Methods for Neural Machine Translation
• Domain specialization: a post-training domain adaptation for Neural Machine Translation
• Domain Control for Neural Machine Translation
• An Arabic-Hebrew parallel corpus of TED talks

Cited by in corpus (12)

• A Survey of Domain Adaptation for Neural Machine Translation
• MetaMT,a MetaLearning Method Leveraging Multiple Domain Data for Low Resource Machine Translation
• Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection
• Multilingual Multi-Domain Adaptation Approaches for Neural Machine Translation
• Machine Translation System Selection from Bandit Feedback
• Learning a Multi-Domain Curriculum for Neural Machine Translation
• Addressing Zero-Resource Domains Using Document-Level Context in Neural Machine Translation
• Neural Machine Translation: A Review and Survey
• Dynamically Composing Domain-Data Selection with Clean-Data Selection by "Co-Curricular Learning" for Neural Machine Translation
• Reinforcement Learning based Curriculum Optimization for Neural Machine Translation
• Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training
• Iterative Dual Domain Adaptation for Neural Machine Translation