Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models

Ning Ding, Yujia Qin, Guang Yang +17

Despite the success, the process of fine-tuning large-scale PLMs brings prohibitive adaptation costs. In fact, fine-tuning all the parameters of a colossal model and retaining sepa…