Latent Prototype Routing: Achieving Near-Perfect Load Balancing in Mixture-of-Experts

Jiajie Yang

Mixture-of-Experts (MoE) architectures have emerged as a key strategy for scaling large language models (LLMs) efficiently. However, current MoE systems suffer from severe load imb…