21 citations · 72 across the 20 of their papers we have counts for
3 papers · 1 filter
SmartGen: Seamless Disaggregated LLM Inference with Selective KV Cache Transfer
Xuchuan Luo, Jiacheng Shen, Xin Wang +1
Disaggregating the prefill and decoding stages of large language model (LLM) inference into two separate sets of nodes is widely adopted in today's LLM serving systems. However, su…
Ditto: An Elastic and Adaptive Memory-Disaggregated Caching System
Jiacheng Shen, Pengfei Zuo, Xuchuan Luo +5
In-memory caching systems are fundamental building blocks in cloud services. However, due to the coupled CPU and memory on monolithic servers, existing caching systems cannot elast…
FUSEE: A Fully Memory-Disaggregated Key-Value Store (Extended Version)
Jiacheng Shen, Pengfei Zuo, Xuchuan Luo +4
Distributed in-memory key-value (KV) stores are embracing the disaggregated memory (DM) architecture for higher resource utilization. However, existing KV stores on DM employ a sem…