Gaussian Mixture Attention: Linear-Time Sequence Mixing via Probabilistic Latent Routing

Yongchao Huang, Hassan Raza

The dense token-to-token interaction pattern of standard dot-product attention remains a central bottleneck in scaling Transformer architectures to long contexts. We introduce \tex…