Rank-Efficient LoRA via Joint Tangent-Space Optimization under Isotropic Curvature

Zihan Zhu, Zhehang Du, Xuyang Chen +5

Low-Rank Adaptation (LoRA) is an effective approach for adapting large pretrained models by learning low-rank weight updates. In practice, the LoRA rank is used to control an adapt…