EarlySelectKV: Anticipatory KV Selection via Cross-Layer Query Proxies

Yuhao Liu, Yuehao Li, Linghe Zhang +4

Query-conditioned sparse KV selection reduces long-context inference cost by attending to a subset of historical KV entries at each decoding step. Existing selectors must wait for…