13 citations · 46 across the 6 of their papers we have counts for
Showing cs.ARShow all
2 papers · 1 filter
cs.AR2026
CacheSmith: Accurate RAG Serving under Latency SLOs with Adaptive KV Reuse
Xinyue Ma, Damin Lee, Aashaka Shah +1
This paper presents CacheSmith, a KV cache reuse system for RAG prefill that maximizes quality-gated SLO attainment. While context caching lowers prefill latency by reusing precomp…
cs.AR2023★ 6 cited
Splitwise: Efficient generative LLM inference using phase splitting
Pratyush Patel, Esha Choukse, Chaojie Zhang +4
Recent innovations in generative large language models (LLMs) have made their applications and use-cases ubiquitous. This has led to large-scale deployments of these models, using…