Dysco: Dynamic Subspace Boosting to Mitigate LoRA Interference in Federated Learning

Haobo Zhang, Jiankun Wang, Suraj Rajendran +5

Federated fine-tuning of large pre-trained models increasingly relies on Low-Rank Adaptation (LoRA) to reduce communication and computation, but heterogeneous clients can make adap…