Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning

Naibin Gu, Peng Fu, Xiyu Liu +3

Parameter-efficient fine-tuning (PEFT) has emerged as the predominant technique for fine-tuning in the era of large language models. However, existing PEFT methods still have inade…