Gradual Fine-Tuning for Low-Resource Domain Adaptation
arXiv:2103.02205
Abstract
Fine-tuning is known to improve NLP models by adapting an initial model trained on more plentiful but less domain-salient examples to data in a target domain. Such domain adaptation is typically done using one stage of fine-tuning. We demonstrate that gradually fine-tuning in a multi-stage process can yield substantial further gains and can be applied without modifying the model or learning objective.
arXiv commentAdapt-NLP, EACL 2021
References in corpus (5)
- Uncertainty-based Continual Learning with Adaptive Regularization
- Improving Neural Machine Translation Robustness via Data Augmentation: Beyond Back Translation
- SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking
- Exploiting Out-of-Domain Parallel Data through Multilingual Transfer Learning for Low-Resource Neural Machine Translation
- Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpus