ARC-KV: Amortizing Anchor Search for Reconstruction-Based KV Cache Compaction

Zheyu Shen, Guanhua Wang, Dezhan Tu +5

Long-context large language model inference is bottlenecked by KV caches that grow linearly with sequence length. This burden is especially severe for long, reusable context prefix…