Showing 2025Show all
2 papers · 1 filter
cs.LG2025
Statistical Advantage of Softmax Attention: Insights from Single-Location Regression
O. Duranthon, P. Marion, C. Boyer +2
Large language models rely on attention mechanisms with a softmax activation. Yet the dominance of softmax over alternatives (e.g., component-wise or linear) remains poorly underst…
stat.ML2025
On the Existence of Consistent Adversarial Attacks in High-Dimensional Linear Classification
Matteo Vilucchio, Lenka Zdeborová, Bruno Loureiro
What fundamentally distinguishes an adversarial attack from a misclassification due to limited model expressivity or finite data? In this work, we investigate this question in the…