Efficient Training of Large-Scale AI Models Through Federated Mixture-of-Experts: A System-Level Approach

Xiaobing Chen, Boyang Zhang, Xiangwei Zhou +4

The integration of Federated Learning (FL) and Mixture-of-Experts (MoE) presents a compelling pathway for training more powerful, large-scale artificial intelligence models (LAMs)…