Remoe: Towards Efficient and Low-Cost MoE Inference in Serverless Computing

Wentao Liu, Yuhao Hu, Ruiting Zhou +2

Mixture-of-Experts (MoE) has become a dominant architecture in large language models (LLMs) due to its ability to scale model capacity via sparse expert activation. Meanwhile, serv…