Trigger the Straggler: Load Hijack on Mixture-of-Experts LLMs

Rui Zhang, Wenbo Jiang, Hongwei Li +4

Expert parallelism (EP) is a common strategy for serving large Mixture-of-Experts (MoE) models across multiple GPUs by distributing experts among devices. Router decisions then det…