3 papers
cs.DC2026
Parallelizing Tool Execution and LLM Generation for Low-Latency Agent Serving
Yifan Sui, Han Zhao, Rui Ma +6
LLM-powered agents execute tasks through a sequential loop of model generation and tool execution. Today's serving systems serialize this loop, leaving tool latency exposed on the…
cs.OS2026
Tutti: Making SSD-Backed KV Cache Practical for Long-Context LLM Serving
Shi Qiu, Yifan Hu, Xintao Wang +6
LLM serving relies on prefix caching to improve inference performance. As growing contexts push key-value (KV) cache footprint far beyond GPU HBM and CPU DRAM capacity, KV cache is…
cs.NI2025
Swift: Rethinking RDMA Control Plane for Elastic Computing
Junxue Zhang, Han Tian, Xinyang Huang +5
Elastic computing enables dynamic scaling to meet workload demands, and Remote Direct Memory Access (RDMA) enhances this by providing high-throughput, low-latency network communica…