LoRA${}^2$ : Multi-Scale Low-Rank Approximations for Fine-Tuning Large Language Models

Jia-Chen Zhang, Yu-Jie Xiong, He-Xi Qiu +2

Fine-tuning large language models (LLMs) with high parameter efficiency for downstream tasks has become a new paradigm. Low-Rank Adaptation (LoRA) significantly reduces the number…