More Than a Quick Glance: Overcoming the Greedy Bias in KV-Cache Compression

Aryan Sood, Tanvi Sharma, Vansh Agrawal

While Large Language Models (LLMs) can theoretically support extensive context windows, their actual deployment is constrained by the linear growth of Key-Value (KV) cache memory.…