Who Should Own the Expert Cache? Kernel-Managed Tiering for Trillion-Parameter MoE Inference

Yuan Si, Yufeng Lin, Daming Li +1

Mixture-of-experts models whose expert pools dwarf DRAM force every serving system to contain a cache, yet existing systems typically implement this cache in user space using exper…