PeekKV: The First Content Token Already Knows Which Heads Matter

Minyoung Go, Yongjun Park

The size of the KV cache grows linearly with context length, making it a major memory bottleneck in LLM serving. Motivated by recent progress in identifying the functional roles of…