Towards Efficient Key-Value Cache Management for Prefix Prefilling in LLM Inference

Yue Zhu, Hao Yu, Chen Wang +2

The increasing adoption of large language models (LLMs) with extended context windows necessitates efficient Key-Value Cache (KVC) management to optimize inference performance. Inf…