Cluster-Driven Expert Pruning for Mixture-of-Experts Large Language Models

Hongcheng Guo, Juntao Yao, Boyang Wang +5

Mixture-of-Experts (MoE) architectures have emerged as a promising paradigm for scaling large language models (LLMs) with sparse activation of task-specific experts. Despite their…