A Spatio-Temporal Expert Prefetching Framework for Efficient MoE-based LLM Inference

Yingnan Zhao, Razvan Bunescu, Ahmed Louri +2

Mixture-of-Experts (MoE) based large language models (LLMs), such as Qwen and DeepSeek, have recently emerged as an effective approach to improving model capacity without proportio…