Distribution-Consistent Inference for Dynamic Sparse Mixture-of-Experts

Dohyeon Kim, Bedionita Soro, Sung Ju Hwang

Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling model capacity while preserving efficient inference in large foundation models. However, most…