FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework

Jianian Zhu, Hang Wu, Haojie Wang +4

Multi-modal Large Language Models (MLLMs) serving systems commonly employ KV-cache compression to reduce memory footprint. However, existing compression methods introduce significa…