ObjectCache: Layerwise Object-Storage Retrieval for KV Cache Reuse

Yu Zhu, Aditya Dhakal, Yunming Xiao +2

Prefix KV caching has become a key mechanism in LLM serving: it reduces time to first token (TTFT) by avoiding redundant computation across requests that share a prefix (i.e., the…