Fine-Tuning Pre-Trained Language Models Effectively by Optimizing Subnetworks Adaptively

Haojie Zhang, Ge Li, Jia Li +3

Large-scale pre-trained language models have achieved impressive results on a wide range of downstream tasks recently. However, fine-tuning an extremely large-scale pre-trained lan…