When Is Rank-1 Enough? Geometry-Guided Initialization for Parameter-Efficient Fine-Tuning

Haoran Zhao, Soyeon Caren Han, Eduard Hovy

Parameter-efficient fine-tuning (PEFT) is a standard way to adapt multimodal large language models, yet extremely low-rank settings -- especially rank-1 LoRA -- are often unstable.…