Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation
arXiv:2010.02705
Abstract
We propose a method to automatically generate a domain- and task-adaptive maskings of the given text for self-supervised pre-training, such that we can effectively adapt the language model to a particular target task (e.g. question answering). Specifically, we present a novel reinforcement learning-based framework which learns the masking policy, such that using the generated masks for further pre-training of the target language model helps improve task performance on unseen texts. We use off-policy actor-critic with entropy regularization and experience replay for reinforcement learning, and propose a Transformer-based policy network that can consider the relative importance of words in a given text. We validate our Neural Mask Generator (NMG) on several question answering and text classification datasets using BERT and DistilBERT as the language models, on which it outperforms rule-based masking strategies, by automatically learning optimal adaptive maskings.
arXiv comment19 pages, 9 figures, EMNLP 2020
References in corpus (11)
- Distributed Representations of Words and Phrases and their Compositionality
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Decoupled Weight Decay Regularization
- BioBERT: a pre-trained biomedical language representation model for biomedical text mining
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Playing Atari with Deep Reinforcement Learning
- Trust Region Policy Optimization
- DARTS: Differentiable Architecture Search
- Theoretical Models of Learning to Learn
- ERNIE: Enhanced Representation through Knowledge Integration
- Domain Adaptive Transfer Learning with Specialist Models