Task-KV: Task-aware KV Cache Optimization via Semantic Differentiation of Attention Heads

Xingyang He, Jie Liu, Shaowei Chen

KV cache is a widely used acceleration technique for large language models (LLMs) inference. However, its memory requirement grows rapidly with input length. Previous studies have…