Parameter-Efficient Tuning Makes a Good Classification Head

Zhuoyi Yang, Ming Ding, Yanhui Guo +2

In recent years, pretrained models revolutionized the paradigm of natural language understanding (NLU), where we append a randomly initialized classification head after the pretrai…