Unifying Linear-Time Attention via Latent Probabilistic Modelling

Rares Dolga, Lucas Maystre, Marius Cobzarenco +1

Transformers have achieved state-of-the-art results across a range of domains, but their quadratic attention mechanism poses significant challenges for long-sequence modelling. Rec…