Beyond Uniform Experts: Cost-Aware Expert Execution for Efficient Multi-Device MoE Inference

Hui Zang, Pengfei Xia, Hong Liu +5

Mixture-of-Experts (MoE) architectures enable language models to achieve unprecedented scale via sparse activation. However, their inference performance is often limited by data mo…