Mixture-of-Experts Serving

Zhiyi Huang, Qinpei Lou, Tao Xiao

Mixture-of-Experts (MoE) models route each token to only a few expert networks, distributing the serving load across experts whose popularity shifts over time. A serving system mus…