CIPHER-MoE: Balancing Efficiency and Routing Fidelity in Trillion-Scale MoE Training

Jing Li, Jian Meng, Yingmeng Gao +8

Mixture-of-Experts (MoE) has been widely adopted in recent large language model (LLM) architectures. However, scaling up MoE in LLM training introduces system-level challenges on t…