2 papers
cs.DC2026
HiNa-MoE: High-Performance, Non-Intrusive MoE Inference on CPUs with Matrix Engines
Weiling Yang, Junwen Zhang, Dezun Dong +4
Mixture-of-Experts (MoE) inference is increasingly deployed in local and on-premise environments, where expert parameters often exceed GPU memory capacity. In latency-sensitive, lo…
cs.DC2026
Physically Partitioned KVCache Format for CPU--GPU Load Balancing in MoE Inference
Enda Yu, Dezun Dong, Xiangke Liao
Single-GPU long-context inference with Mixture-of-Experts (MoE) models requires spilling the key-value cache (KVCache) to CPU memory. The spilled KV serves two complementary purpos…