works on
distributed cache 1hybrid sliding window attention 1kvcache optimization 1mixture-of-experts 1multimodal inference 1

From the 1 of 8 linked papers with an AI index.

activity
20242026
collaborators
Showing cs.ETShow all

1 paper · 1 filter

cs.ET2025

Towards Efficient Key-Value Cache Management for Prefix Prefilling in LLM Inference

Yue Zhu, Hao Yu, Chen Wang +2

The increasing adoption of large language models (LLMs) with extended context windows necessitates efficient Key-Value Cache (KVC) management to optimize inference performance. Inf…