2 papers
cs.LG2026
2Mamba2Furious: Linear in Complexity, Competitive in Accuracy
Gabriel Mongaras, Eric C. Larson
Linear attention transformers have become a strong alternative to softmax attention due to their efficiency. However, linear attention tends to be less expressive and results in re…
cs.LG2025
On the Expressiveness of Softmax Attention: A Recurrent Neural Network Perspective
Gabriel Mongaras, Eric C. Larson
Since its introduction, softmax attention has become the backbone of modern transformer architectures due to its expressiveness and scalability across a wide range of tasks. Howeve…