4 papers
WiCi: Wireless GPU Computing Infrastructure
Yibin Shen, Wei Li, Kaiqiang Xu +1
LLM inference applications are gaining significant traction. The demand for inference is growing exponentially, and the GPU usage of inference is increasingly surpassing that of tr…
Tutti: Making SSD-Backed KV Cache Practical for Long-Context LLM Serving
Shi Qiu, Yifan Hu, Xintao Wang +6
LLM serving relies on prefix caching to improve inference performance. As growing contexts push key-value (KV) cache footprint far beyond GPU HBM and CPU DRAM capacity, KV cache is…
Parallelizing Tool Execution and LLM Generation for Low-Latency Agent Serving
Yifan Sui, Han Zhao, Rui Ma +6
LLM-powered agents execute tasks through a sequential loop of model generation and tool execution. Today's serving systems serialize this loop, leaving tool latency exposed on the…
Swift: Rethinking RDMA Control Plane for Elastic Computing
Junxue Zhang, Han Tian, Xinyang Huang +5
Elastic computing enables dynamic scaling to meet workload demands, and Remote Direct Memory Access (RDMA) enhances this by providing high-throughput, low-latency network communica…