The Scaling Law for LoRA Base on Mutual Information Upper Bound

Jing Zhang, Hui Gao, Peng Zhang +3

LoRA (Low-Rank Adaptation) is a widely used model fine-tuning method. In fine-tuning, the law among model performance, model parameters, and data complexity has been a focal issue…