SlimCaching: Edge Caching of Mixture-of-Experts for Distributed Inference

Qian Chen, Xianhao Chen, Kaibin Huang

Mixture-of-Experts (MoE) models improve the scalability of large language models (LLMs) by activating only a small subset of relevant experts per input. However, the sheer number o…