FluxMoE: Decoupling Expert Residency for High-Performance MoE Serving

Qingxiu Liu, Yongchao He, Runhan Jiang +4

Mixture-of-Experts (MoE) models have become mainstream for scaling language models to hundreds of billions of expert parameters. Despite sparse expert activation, existing inferenc…