CacheSmith: Accurate RAG Serving under Latency SLOs with Adaptive KV Reuse

Xinyue Ma, Damin Lee, Aashaka Shah +1

This paper presents CacheSmith, a KV cache reuse system for RAG prefill that maximizes quality-gated SLO attainment. While context caching lowers prefill latency by reusing precomp…