Fast Domain Adaptation for Neural Machine Translation

Abstract

Neural Machine Translation (NMT) is a new approach for automatic translation of text from one human language into another. The basic concept in NMT is to train a large Neural Network that maximizes the translation performance on a given parallel corpus. NMT is gaining popularity in the research community because it outperformed traditional SMT approaches in several translation tasks at WMT and other evaluation tasks/benchmarks at least for some language pairs. However, many of the enhancements in SMT over the years have not been incorporated into the NMT framework. In this paper, we focus on one such enhancement namely domain adaptation. We propose an approach for adapting a NMT system to a new domain. The main idea behind domain adaptation is that the availability of large out-of-domain training data and a small in-domain training data. We report significant gains with our proposed method in both automatic metrics and a human subjective evaluation metric on two language pairs. With our adaptation method, we show large improvement on the new domain while the performance of our general domain only degrades slightly. In addition, our approach is fast enough to adapt an already trained system to a new domain within few hours without the need to retrain the NMT model on the combined data which usually takes several days/weeks depending on the volume of the data.

References in corpus (2)

• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Empirical Methods for Compound Splitting

Cited by in corpus (15)

• An Empirical Comparison of Simple Domain Adaptation Methods for Neural Machine Translation
• Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies
• Exploring Benefits of Transfer Learning in Neural Machine Translation
• Multilingual Multi-Domain Adaptation Approaches for Neural Machine Translation
• Domain Adaptation of Neural Machine Translation by Lexicon Induction
• Curriculum Learning for Domain Adaptation in Neural Machine Translation
• Improving Robustness and Generality of NLP Models Using Disentangled Representations
• Word-based Domain Adaptation for Neural Machine Translation
• Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation
• Go From the General to the Particular: Multi-Domain Translation with Domain Transformation Networks
• Meta-Learning for Few-Shot NMT Adaptation
• Reinforcement Learning based Curriculum Optimization for Neural Machine Translation
• Dynamically Composing Domain-Data Selection with Clean-Data Selection by "Co-Curricular Learning" for Neural Machine Translation
• Iterative Dual Domain Adaptation for Neural Machine Translation
• Personalizing Grammatical Error Correction: Adaptation to Proficiency Level and L1