disaggregated serving 1kv cache transfer 1large language model inference 1network bandwidth optimization 1prefill-decoding separation 1
From the 1 of 3 linked papers with an AI index.
3 papers
cs.DC2026
SmartGen: Seamless Disaggregated LLM Inference with Selective KV Cache Transfer
Xuchuan Luo, Jiacheng Shen, Xin Wang +1
The paper introduces SmartGen, a system that reduces network overhead in disaggregated large language model inference by selectively transferring only essential key‑value cache ent…
cs.OS2026
EdgeFlow: Fast Cold Starts for LLMs on Mobile Devices
Yongsheng Yan, Jiacheng Shen, Xuchuan Luo +1
Deploying large language models (LLMs) on mobile devices is an emerging trend to enable data privacy and offline accessibility of LLM applications. Modern mobile neural processing…
cs.DC2026
CIDER: Boosting Memory-Disaggregated Key-Value Stores with Pessimistic Synchronization
Yuxuan Du, Xuchuan Luo, Xin Wang +2
Memory-disaggregated key-value (KV) stores suffer from a severe performance bottleneck due to their I/O redundancy issues. A huge amount of redundant I/Os are generated when synchr…