Regularizing Subspace Redundancy of Low-Rank Adaptation

Yue Zhu, Haiwen Diao, Shang Gao +8

Low-Rank Adaptation (LoRA) and its variants have delivered strong capability in Parameter-Efficient Transfer Learning (PETL) by minimizing trainable parameters and benefiting from…