Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts

Yongxiang Hua, Haoyu Cao, Zhou Tao +4

Sparse Mixture of Experts (sMoE) has become a pivotal approach for scaling large vision-language models, offering substantial capacity while maintaining computational efficiency th…