5 papers
Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networks
Jie Huang, Bruno Loureiro, Stefano Sarao Mannelli
We study the population loss landscape of two-layer ReLU networks of the form in a realisable teacher-student setting with Gaussian covaria…
A Noise Sensitivity Exponent Controls Large Statistical-to-Computational Gaps in Single- and Multi-Index Models
Leonardo Defilippis, Florent Krzakala, Bruno Loureiro +1
Understanding when learning is statistically possible yet computationally hard is a central challenge in high-dimensional statistics. In this work, we investigate this question in…
Statistical Advantage of Softmax Attention: Insights from Single-Location Regression
O. Duranthon, P. Marion, C. Boyer +2
Large language models rely on attention mechanisms with a softmax activation. Yet the dominance of softmax over alternatives (e.g., component-wise or linear) remains poorly underst…
On the existence of consistent adversarial attacks in high-dimensional linear classification
Matteo Vilucchio, Lenka Zdeborová, Bruno Loureiro
What fundamentally distinguishes an adversarial attack from a misclassification due to limited model expressivity or finite data? In this work, we investigate this question in the…
On the Geometry of Regularization in Adversarial Training: High-Dimensional Asymptotics and Generalization Bounds
Matteo Vilucchio, Nikolaos Tsilivis, Bruno Loureiro +1
Regularization, whether explicit in terms of a penalty in the loss or implicit in the choice of algorithm, is a cornerstone of modern machine learning. Indeed, controlling the comp…