ElasticMoE: An Efficient Auto Scaling Method for Mixture-of-Experts Models

Gursimran Singh, Timothy Yu, Haley Li +7

Mixture-of-Experts (MoE) models promise efficient scaling of large language models (LLMs) by activating only a small subset of experts per token, but their parallelized inference p…