MoR: Mixture of Ranks for Low-Rank Adaptation Tuning

Chuanyu Tang, Yilong Chen, Zhenyu Zhang +4

Low-Rank Adaptation (LoRA) drives research to align its performance with full fine-tuning. However, significant challenges remain: (1) Simply increasing the rank size of LoRA does…