KVCompose: Efficient Structured KV Cache Compression with Composite Tokens

Dmitry Akulov, Mohamed Sana, Antonio De Domenico +3

Large language models (LLMs) rely on key-value (KV) caches for efficient autoregressive decoding; however, cache size grows linearly with context length and model depth, becoming a…