Tensor Decomposition of Transformer Key-Value Caches: Spectral Structure and Format Comparison

Rahul Krishnan, Volker Schulz

The key-value (KV) cache of autoregressive transformers can be viewed as a fourth-order tensor spanning attention heads, tokens, features, and grouped layers. We measure the singul…