Q-Filters: Leveraging QK Geometry for Efficient KV Cache Compression

Nathan Godey, Alessio Devoto, Yu Zhao +4

Autoregressive language models rely on a Key-Value (KV) Cache, which avoids re-computing past hidden states during generation, making it faster. As model sizes and context lengths…