ReFreeKV: Towards Threshold-Free KV Cache Compression

Xuanfan Ni, Liyan Xu, Chenyang Lyu +6

To reduce memory consumption during LLM inference, a handful of methods have been proposed for KV cache pruning. While these techniques can accomplish lossless memory reduction on…