C${}^2$KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference

Chuheng Du, Junyi Chen, Hanlin Tang +7

Long-context inference is central to modern large language model (LLM) applications such as retrieval-augmented generation and multi-document reasoning. To mitigate the growing inf…