Guiding the Experts: Semantic Priors for Efficient and Focused MoE Routing

Chengxi Min, Wei Wang, Yahui Liu +4

Mixture-of-Experts (MoE) models have emerged as a promising direction for scaling vision architectures efficiently. Among them, Soft MoE improves training stability by assigning ea…