2 papers
math.NA2026
Tensor Decomposition of Transformer Key-Value Caches: Spectral Structure and Format Comparison
Rahul Krishnan, Volker Schulz
The key-value (KV) cache of autoregressive transformers can be viewed as a fourth-order tensor spanning attention heads, tokens, features, and grouped layers. We measure the singul…
cs.LG2026
A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocation
Rahul Krishnan, Volker Schulz
The key-value (KV) cache is the dominant memory bottleneck in long-context language model inference. Existing compression methods apply low-rank factorization or quantization indep…