GoRA: Gradient-driven Adaptive Low Rank Adaptation

Haonan He, Peng Ye, Yuchen Ren +4

Low-Rank Adaptation (LoRA) is a crucial method for efficiently fine-tuning large language models (LLMs), with its effectiveness influenced by two key factors: rank selection and we…