High-Rank Structured Modulation for Parameter-Efficient Fine-Tuning

Yongkang Liu, Xing Li, Mengjie Zhao +7

As the number of model parameters increases, parameter-efficient fine-tuning (PEFT) has become the go-to choice for tailoring pre-trained large language models. Low-rank Adaptation…