GEM: GPU-Variability-Aware Expert to GPU Mapping for MoE Systems

Sourish Wawdhane, Avinash Kumar, Poulami Das

Mixture-of-Expert (MoE) models enable efficient inference by employing smaller experts and activating only a subset of them per token. MoE serving engines distribute experts across…