PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularization

Yao Ni, Shan Zhang, Piotr Koniusz

Parameter-Efficient Fine-Tuning (PEFT) effectively adapts pre-trained transformers to downstream tasks. However, the optimization of tasks performance often comes at the cost of ge…