MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models

Jinsong Shu, Chenyang Wu, Zhongle Xie +2

Key-Value (KV) caching is essential for efficient inference in multimodal large language models (MLLMs), yet its memory footprint grows linearly with context length and becomes a m…