3 papers
cs.LG2025
LUNA: Linear Universal Neural Attention with Generalization Guarantees
Ashkan Shahbazi, Ping He, Ali Abbasi +6
Scaling attention faces a critical bottleneck: the quadratic computational cost of softmax attention, which limits its application in long-sequence domains. Whil…
cs.LG2025
Constrained Sliced Wasserstein Embedding
Navid NaderiAlizadeh, Darian Salehi, Xinran Liu +1
Sliced Wasserstein (SW) distances offer an efficient method for comparing high-dimensional probability measures by projecting them onto multiple 1-dimensional probability distribut…
cs.LG2025
ESPFormer: Doubly-Stochastic Attention with Expected Sliced Transport Plans
Ashkan Shahbazi, Elaheh Akbari, Darian Salehi +3
While self-attention has been instrumental in the success of Transformers, it can lead to over-concentration on a few tokens during training, resulting in suboptimal information fl…