TaRA: Training-Aware Low-Rank Adaptation Initialization

Taehyeon Kim, Eunhyeok Park

Low-Rank Adaptation (LoRA) has become a de facto standard for parameter-efficient fine-tuning (PEFT), yet its performance is highly sensitive to initialization due to the informati…