BrownoutMoE: Structure-Aware Expert Grouping for Efficient and Accurate LLM Web-based Services

Yi Ding, Minxian Xu, Zhengxin Fang +2

Mixture-of-Experts (MoE) large language models (LLMs) are increasingly deployed in Web-facing services, where inference must be both accurate and responsive under bursty demand. Al…