FoRA: Fisher-orthogonal Rank Adaptation for Parameter-Efficient Fine-Tuning

Juneyoung Park, Seongbae Lee, Han-Sang Lee +6

Parameter-efficient fine-tuning(PEFT) has largely focused on LoRA and its accuracy-oriented variants, leaving the original goal of reducing trainable parameters has receivedcompara…