Fisher Mask Nodes for Language Model Merging

Thennal D K, Ganesh Nathan, Suchithra M S

Fine-tuning pre-trained models provides significant advantages in downstream performance. The ubiquitous nature of pre-trained models such as BERT and its derivatives in natural la…