MoE-CORE: Coordinated Expert Offloading and Residency for Memory-Constrained MoE Inference

Ke Yang, Yongji Gao, Xushi Li +15

Sparse expert activation reduces MoE models' computation, yet expert weights can exceed limited device memory. Offloading makes inference feasible on a compact AI appliance but exp…