FISH-Tuning: Enhancing PEFT Methods with Fisher Information

Kang Xue, Ming Dong, Xinhui Tu +1

The rapid growth in the parameter size of Large Language Models (LLMs) has spurred the development of Parameter-Efficient Fine-Tuning (PEFT) methods to mitigate the substantial com…