Compressed Convolutional Attention: Efficient Attention in a Compressed Latent Space

Tomas Figliolia, Nicholas Alonso, Rishi Iyer +2

Multi-headed Attention's (MHA) quadratic compute and linearly growing KV-cache make long-context transformers expensive to train and serve. Prior works such as Grouped Query Attent…