Unraveling LoRA Interference: Orthogonal Subspaces for Robust Model Merging

Haobo Zhang, Jiayu Zhou

Fine-tuning large language models (LMs) for individual tasks yields strong performance but is expensive for deployment and storage. Recent works explore model merging to combine mu…