4 papers
CacheWise: Understanding Workloads and Optimizing KVCache Management for Efficiently Serving LLM Coding Agents
Shubham Tiwari, Tapan Chugh, Nash Rickert +3
Coding agents are a fast-growing LLM application, executing as long-running closed-loop sessions in which LLM generations alternate with external tool calls. Yet, unlike chat workl…
Reducing the GPU Memory Bottleneck with Lossless Compression for ML -- Extended
Aditya K Kamath, Arvind Krishnamurthy, Marco Canini +1
Machine learning (ML) training and inference often process data sets far exceeding GPU memory capacity, forcing them to rely on PCIe for on-demand tensor transfers, causing critica…
VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?
Keisuke Kamahori, Shihang Li, Simon Peter +1
For years, we have built LLM serving systems like any other critical infrastructure: a single general-purpose stack, hand-tuned over many engineer-years, meant to support every mod…
Cloud abstractions for AI workloads
Marco Canini, Theophilus A. Benson, Ricardo Bianchini +4
AI workloads, often hosted in multi-tenant cloud environments, require vast computational resources but suffer inefficiencies due to limited tenant-provider coordination. Tenants l…