disaggregated serving 1kv cache transfer 1large language model inference 1network bandwidth optimization 1prefill-decoding separation 1
From the 1 of 4 linked papers with an AI index.
Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
SmartGen: Seamless Disaggregated LLM Inference with Selective KV Cache Transfer
Xuchuan Luo, Jiacheng Shen, Xin Wang +1
The paper introduces SmartGen, a system that reduces network overhead in disaggregated large language model inference by selectively transferring only essential key‑value cache ent…
cs.DC2026
CIDER: Boosting Memory-Disaggregated Key-Value Stores with Pessimistic Synchronization
Yuxuan Du, Xuchuan Luo, Xin Wang +2
Memory-disaggregated key-value (KV) stores suffer from a severe performance bottleneck due to their I/O redundancy issues. A huge amount of redundant I/Os are generated when synchr…