Hierarchical Adaptive Eviction for KV Cache Management in Multimodal Language Models

Xindian Ma, Yidi Lu, Peng Zhang +1

The integration of visual information into Large Language Models (LLMs) has enabled Multimodal LLMs (MLLMs), but the quadratic memory and computational costs of Transformer archite…