FedWeave: Rethinking the Unit of Specialization in Heterogeneous Federated MoE-LoRA

Donghang Duan, Xu Zheng, Lizong Zhang +2

Federated PEFT enables LLMs to collaboratively adapt to decentralized private data without sharing raw examples. However, task heterogeneity across clients can cause cross-task int…