SPT: Fine-Tuning Transformer-based Language Models Efficiently with Sparsification

Yuntao Gui, Xiao Yan, Peiqi Yin +2

Transformer-based large language models (e.g., BERT and GPT) achieve great success, and fine-tuning, which tunes a pre-trained model on a task-specific dataset, is the standard pra…