PATS: Sensitivity-aware Noisy Learning for Pretrained Language Models

Yupeng Zhang, Hongzhi Zhang, Sirui Wang +2

A wide range of NLP tasks benefit from the fine-tuning of pretrained language models (PLMs). However, a number of redundant parameters which contribute less to the downstream task…