Meta-Learning for Few-Shot NMT Adaptation
arXiv:2004.02745
Abstract
We present META-MT, a meta-learning approach to adapt Neural Machine Translation (NMT) systems in a few-shot setting. META-MT provides a new approach to make NMT models easily adaptable to many target domains with the minimal amount of in-domain data. We frame the adaptation of NMT systems as a meta-learning problem, where we learn to adapt to new unseen domains based on simulated offline meta-training domain adaptation tasks. We evaluate the proposed meta-learning strategy on ten domains with general large scale NMT systems. We show that META-MT significantly outperforms classical domain adaptation when very few in-domain examples are available. Our experiments shows that META-MT can outperform classical fine-tuning by up to 2.5 BLEU points after seeing only 4, 000 translated words (300 parallel sentences).
References in corpus (6)
- Adam: A Method for Stochastic Optimization
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Fast Domain Adaptation for Neural Machine Translation
- fairseq: A Fast, Extensible Toolkit for Sequence Modeling
- Simple, Scalable Adaptation for Neural Machine Translation
- Enhanced Meta-Learning for Cross-lingual Named Entity Recognition with Minimal Resources