Fast Domain Adaptation for Neural Machine Translation

Abstract

Neural Machine Translation (NMT) is a new approach for automatic translation of text from one human language into another. The basic concept in NMT is to train a large Neural Network that maximizes the translation performance on a given parallel corpus. NMT is gaining popularity in the research community because it outperformed traditional SMT approaches in several translation tasks at WMT and other evaluation tasks/benchmarks at least for some language pairs. However, many of the enhancements in SMT over the years have not been incorporated into the NMT framework. In this paper, we focus on one such enhancement namely domain adaptation. We propose an approach for adapting a NMT system to a new domain. The main idea behind domain adaptation is that the availability of large out-of-domain training data and a small in-domain training data. We report significant gains with our proposed method in both automatic metrics and a human subjective evaluation metric on two language pairs. With our adaptation method, we show large improvement on the new domain while the performance of our general domain only degrades slightly. In addition, our approach is fast enough to adapt an already trained system to a new domain within few hours without the need to retrain the NMT model on the combined data which usually takes several days/weeks depending on the volume of the data.

References in corpus (3)

• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Empirical Methods for Compound Splitting
• Vocabulary Manipulation for Neural Machine Translation

Cited by in corpus (36)

• A Survey of Domain Adaptation for Neural Machine Translation
• Curriculum Learning and Minibatch Bucketing in Neural Machine Translation
• Imagination improves Multimodal Translation
• Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translation
• An Empirical Comparison of Simple Domain Adaptation Methods for Neural Machine Translation
• Neural Machine Translation Training in a Multi-Domain Scenario
• Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies
• Exploring Benefits of Transfer Learning in Neural Machine Translation
• A Simple Baseline to Semi-Supervised Domain Adaptation for Machine Translation
• Simple, Scalable Adaptation for Neural Machine Translation
• Multilingual Multi-Domain Adaptation Approaches for Neural Machine Translation
• Domain Adaptation of Neural Machine Translation by Lexicon Induction
• Transductive Data-Selection Algorithms for Fine-Tuning Neural Machine Translation
• Curriculum Learning for Domain Adaptation in Neural Machine Translation
• Improving Robustness and Generality of NLP Models Using Disentangled Representations
• Word-based Domain Adaptation for Neural Machine Translation
• Learning a Multi-Domain Curriculum for Neural Machine Translation
• Addressing Zero-Resource Domains Using Document-Level Context in Neural Machine Translation
• Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation
• Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters
• Meta-Learning for Few-Shot NMT Adaptation
• Rapid Domain Adaptation for Machine Translation with Monolingual Data
• Vocabulary Adaptation for Distant Domain Adaptation in Neural Machine Translation
• Neural Machine Translation: A Review and Survey
• Go From the General to the Particular: Multi-Domain Translation with Domain Transformation Networks
• Dynamically Composing Domain-Data Selection with Clean-Data Selection by "Co-Curricular Learning" for Neural Machine Translation
• Interventional Domain Adaptation
• Revisiting Simple Domain Adaptation Methods in Unsupervised Neural Machine Translation
• Reinforcement Learning based Curriculum Optimization for Neural Machine Translation
• Investigating Catastrophic Forgetting During Continual Training for Neural Machine Translation
• Using Machine Translation to Localize Task Oriented NLG Output
• Personalizing Grammatical Error Correction: Adaptation to Proficiency Level and L1
• Learning Policies for Multilingual Training of Neural Machine Translation Systems
• Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speech
• The ADAPT System Description for the IWSLT 2018 Basque to English Translation Task
• Iterative Dual Domain Adaptation for Neural Machine Translation