GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models

Chen Zheng, Yuhang Cai, Deyi Liu +7

Modern large language models leverage Mixture-of-Experts (MoE) architectures for efficient scaling, but face a critical challenge: functionally similar experts are often selected s…