DiffoRA: Enabling Parameter-Efficient Fine-Tuning via Differential Module Selection

Tangyu Jiang, Haodi Wang, Chun Yuan

The Parameter-Efficient Fine-Tuning (PEFT) methods have been extensively researched for large language models in downstream tasks. Among all the existing approaches, the Low-Rank A…