MoE Router-Guided Clustering for Heterogeneous Federated Instruction Tuning

Ankita Sharma, Bahar Farahani, Sanaz Rahimi Moosavi +3

Federated instruction fine-tuning enables Large Language Models (LLMs) to adapt to decentralized, privacy-sensitive data without requiring data sharing. Recent Mixture-of-Experts (…