TGV-KV: Text-Grounded KV Eviction for Vision-Language Models

Jizhihui Liu, Ruizi Han, Miao Zhang +4

Vision-Language Models (VLMs) inherit the auto-regressive generation paradigm and cache the keys and values (KV) of all previous tokens to accelerate inference, resulting in memory…