6 citations · 9 across the 11 of their papers we have counts for
Showing 2026 · cs.AIShow all
2 papers · 2 filters
cs.AI2026
A Year in LLM Serving: Workload Evolution, Caching and Load-Balancing
William Nixon, Jon Durbin, Florian Standhartinger +2
Large Language Model (LLM) serving has become a critical cloud workload, and realistic traces are essential for motivating and benchmarking serving systems. However, existing LLM s…
cs.AI2026
HeadWiseKV: Budgeted Per-Head Cache Residency for Hybrid Long-Context Language Models
Renjie Xie, Juncheng Yang, Aoting Hu +4
Long-context inference retains a growing key--value (KV) cache during decoding, which consumes substantial GPU memory and can reduce generation throughput. This bottleneck remains…