FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models

Fan Mo, Yuxuan Han, Geng Zhang +2

Mixture-of-Experts (MoE) language models scale model ability with sparsely activated experts, making this architecture a standard recipe for modern large models. However, sparse ac…