3 papers
cs.LG2026
QUOKA: Query-Oriented KV Selection For Efficient LLM Prefill
Dalton Jones, Junyoung Park, Matthew Morse +3
We present QUOKA: Query-oriented KV selection for efficient attention, a training-free and hardware agnostic sparse attention algorithm for accelerating transformer inference under…
cs.AI2026
KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments
Junyoung Park, Dalton Jones, Matthew J Morse +3
We demonstrate that geometrically distinctive keys during LLM inference tend to have high attention scores. Based on the phenomenon we propose KeyDiff, a training-free KV cache evi…
cs.LG2025
CAOTE: KV Cache Selection for LLMs via Attention Output Error-Based Token Eviction
Raghavv Goel, Junyoung Park, Mukul Gagrani +5
While long context support of large language models has extended their abilities, it also incurs challenges in memory and compute which becomes crucial bottlenecks in resource-rest…