SemantiCache: Efficient KV Cache Compression via Semantic Chunking and Clustered Merging

Shunlong Wu, Hai Lin, Shaoshen Chen +5

Existing KV cache compression methods generally operate on discrete tokens or non-semantic chunks. However, such approaches often lead to semantic fragmentation, where linguistical…