Adaptive Utilization of Low-Rank Adaptation via Conditioned Gating

Guang Yang, Changhao Guan, Chao Huang +2

Low-Rank Adaptation (LoRA) achieves parameter-efficient fine-tuning by constraining model updates to a low-rank subspace and has been widely used in practice. However, LoRA typical…