Showing 2025Show all
2 papers · 1 filter
cs.ET2025
Towards Efficient Key-Value Cache Management for Prefix Prefilling in LLM Inference
Yue Zhu, Hao Yu, Chen Wang +2
The increasing adoption of large language models (LLMs) with extended context windows necessitates efficient Key-Value Cache (KVC) management to optimize inference performance. Inf…
cs.DC2025
Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inference
Pol G. Recasens, Ferran Agullo, Yue Zhu +5
Large language models have been widely adopted across different tasks, but their auto-regressive generation nature often leads to inefficient resource utilization during inference.…