DenseLoRA: Dense Low-Rank Adaptation of Large Language Models

Lin Mu, Xiaoyu Wang, Li Ni +4

Low-rank adaptation (LoRA) has been developed as an efficient approach for adapting large language models (LLMs) by fine-tuning two low-rank matrices, thereby reducing the number o…