Delta-LoRA: Fine-Tuning High-Rank Parameters with the Delta of Low-Rank Matrices

Bojia Zi, Xianbiao Qi, Lingzhi Wang +3

In this paper, we present Delta-LoRA, which is a novel parameter-efficient approach to fine-tune large language models (LLMs). In contrast to LoRA and other low-rank adaptation met…