Not All LoRA Parameters Are Essential: Insights on Inference Necessity

Guanhua Chen, Yutong Yao, Ci-Jun Gao +3

Current research on LoRA primarily focuses on minimizing the number of fine-tuned parameters or optimizing its architecture. However, the necessity of all fine-tuned LoRA layers du…