2 papers
cs.DC2026
OasisKV: Scaling In-Decode KV Cache Beyond HBM with Lookahead Sparse Prefetching
Can Xiao, Sukmin Cho, Junbong We +7
Large language model (LLM) inference serving is increasingly constrained by memory rather than compute. As long-context and long-form reasoning workloads become more prevalent, the…
cs.DC2026
ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
Sangjin Choi, Sukmin Cho, Yifan Xiong +3
Equal load does not mean equal decode latency in prefill-decode (PD) disaggregated mixture-of-experts (MoE) serving. In bandwidth-bound decode, more distinct experts mean more weig…