LoRA-DA: Data-Aware Initialization for Low-Rank Adaptation via Asymptotic Analysis

Qingyue Zhang, Chang Chu, Tianren Peng +4

LoRA has become a widely adopted method for PEFT, and its initialization methods have attracted increasing attention. However, existing methods have notable limitations: many metho…