STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control

Priyansh Bhatnagar, Ashkan Moradifirouzabadi, Se-Hyun Yang +3

Low-rank projection has emerged as a promising approach for compressing the KV cache by exploiting hidden-dimension redundancy. However, prior methods rely on fixed or heuristic ra…