Expert-as-a-Service: Towards Efficient, Scalable, and Robust Large-scale MoE Serving

Ziming Liu, Boyu Tian, Guoteng Wang +15

Mixture-of-Experts (MoE) models challenge serving infrastructures with dynamic, sparse expert utilization, causing instability on conventional systems designed for dense architectu…