KV Admission: Learning What to Write for Efficient Long-Context LLM Inference

Yen-Chieh Huang, Pi-Cheng Hsiu, Rui Fang +1

Long-context LLM inference is bottlenecked by the quadratic attention complexity and linear Key-Value (KV) cache growth. Prior approaches mitigate this via post-hoc selection or ev…