LouisKV: Efficient KV Cache Retrieval for Long Input-Output Sequences

Wenbo Wu, Qingyi Si, Xiurui Pan +2

While Key-Value (KV) cache succeeds in reducing redundant computations in auto-regressive models, it introduces significant memory overhead, limiting its practical deployment in lo…