Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting

Abstract

Deep pretrained language models have achieved great success in the way of pretraining first and then fine-tuning. But such a sequential transfer learning paradigm often confronts the catastrophic forgetting problem and leads to sub-optimal performance. To fine-tune with less forgetting, we propose a recall and learn mechanism, which adopts the idea of multi-task learning and jointly learns pretraining tasks and downstream tasks. Specifically, we propose a Pretraining Simulation mechanism to recall the knowledge from pretraining tasks without data, and an Objective Shifting mechanism to focus the learning on downstream tasks gradually. Experiments show that our method achieves state-of-the-art performance on the GLUE benchmark. Our method also enables BERT-base to achieve better performance than directly fine-tuning of BERT-large. Further, we provide the open-source RecAdam optimizer, which integrates the proposed mechanisms into Adam optimizer, to facility the NLP community.

References in corpus (13)

• Distilling the Knowledge in a Neural Network
• Decoupled Weight Decay Regularization
• Overcoming catastrophic forgetting in neural networks
• An Overview of Multi-Task Learning in Deep Neural Networks
• Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
• Continual Learning Through Synaptic Intelligence
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• Overcoming Catastrophic Forgetting by Incremental Moment Matching
• Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks
• Less-forgetting Learning in Deep Neural Networks
• Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models
• A Multi-Task Learning Framework for Overcoming the Catastrophic Forgetting in Automatic Speech Recognition
• Forget Me Not: Reducing Catastrophic Forgetting for Domain Adaptation in Reading Comprehension

Cited by in corpus (4)

• Transferability of Natural Language Inference to Biomedical Question Answering
• On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation
• CausalBERT: Injecting Causal Knowledge Into Pre-trained Models with Minimal Supervision
• Investigation of Practical Aspects of Single Channel Speech Separation for ASR