Democratizing MoE inference on commodity GPUs with CoMoE

Ruwen Fan, Yuezhi Zu, Minhui Xie +6

Deploying Mixture-of-Experts (MoE) models relies heavily on Expert Parallelism, which generates intense inter-GPU communication. Consequently, state-of-the-art inference systems re…