Task-Aware Federated Fine-Tuning for MoE-based Large Language Models

Tingqi Wang, Hongyu Ke, Haoxin Wang +2

Mixture-of-Experts (MoE) has become a widely adopted architecture for Large Language Models (LLMs), as it improves model capacity while limiting computational overhead through spar…