KVReviver: Reversible KV Cache Compression with Sketch-Based Token Reconstruction

Aomufei Yuan, Zhiming Wang, Ruijie Miao +8

As the context length of current large language models (LLMs) rapidly increases, the memory demand for the Key-Value (KV) cache is becoming a bottleneck for LLM deployment and batc…