3 papers
cs.LG2026
ScoutAttention: Efficient KV Cache Offloading via Layer-Ahead CPU Pre-computation for LLM Inference
Qiuyang Zhang, Kai Zhou, Ding Tang +5
Large language models encounter critical GPU memory capacity constraints during long-context inference, where KV cache memory consumption severely limits decode batch sizes. While…
cs.DC2025
Scalable Distributed Vector Search via Accuracy Preserving Index Construction
Yuming Xu, Qianxi Zhang, Qi Chen +8
Scaling Approximate Nearest Neighbor Search (ANNS) to billions of vectors requires distributed indexes that balance accuracy, latency, and throughput. Yet existing index designs st…
cs.DC2025
DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving
Chaoyi Ruan, Yinhe Chen, Dongqi Tian +4
LLM inference must meet strict latency SLOs (e.g., 100 ms P99 time-between-tokens) while maximizing goodput. Yet, real-world variability in prompt and response lengths skews comput…