MoETuner: Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing

Seokjin Go, Divya Mahajan

Mixture-of-Experts (MoE) model architecture has emerged as a promising solution for scaling transformer models efficiently, offering sparse activation that reduces computational co…