LoSiA: Efficient High-Rank Fine-Tuning via Subnet Localization and Optimization

Xujia Wang, Yunjia Qi, Bin Xu

Parameter-Efficient Fine-Tuning (PEFT) methods, such as LoRA, significantly reduce the number of trainable parameters by introducing low-rank decomposition matrices. However, exist…