FAD-KV: Adaptive Dictionary KV Caching for Efficient Reasoning Decoding

Yuxuan Zhu, David H. Yang, Parisa Shiravani +5

Key-value (KV) caching is a core mechanism in large language models, where intermediate key and value representations are stored to accelerate autoregressive decoding. As generatio…