Progtuning: Progressive Fine-tuning Framework for Transformer-based Language Models

Xiaoshuang Ji, Zhendong Zhao, Xiaojun Chen +2

Fine-tuning is a promising technique for leveraging Transformer-based language models in downstream tasks. As model sizes continue to grow, updating all model parameters becomes in…