Faster MoE LLM Inference for Extremely Large Models

Haoqi Yang, Luohe Shi, Qiwei Li +5

Sparse Mixture of Experts (MoE) large language models (LLMs) are gradually becoming the mainstream approach for ultra-large-scale models. Existing optimization efforts for MoE mode…