HAP: Hybrid Adaptive Parallelism for Efficient Mixture-of-Experts Inference

Haoran Lin, Xianzhi Yu, Kang Zhao +7

Current inference systems for Mixture-of-Experts (MoE) models primarily employ static parallelization strategies. However, these static approaches cannot consistently achieve optim…