Multi-head Temporal Latent Attention

Keqi Deng, Philip C. Woodland

While Transformer self-attention offers strong parallelism, the Key-Value (KV) cache grows linearly with sequence length and becomes a bottleneck for inference efficiency. Multi-he…