5 papers
Multi-layer State Evolution Under Random Convolutional Design
Mara Daniels, Cédric Gerbelot, Cédric Gerbelot +3
Signal recovery under generative neural network priors has emerged as a promising direction in statistical inference and computational imaging. Theoretical analysis of reconstructi…
How Two-Layer Neural Networks Learn, One (Giant) Step at a Time
Yatin Dandi, Florent Krzakala, Bruno Loureiro +2
For high-dimensional Gaussian data, we investigate theoretically how the features of a two-layer neural network adapt to the structure of the target function through a few large ba…
A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs
Kasimir Tanner, Matteo Vilucchio, Bruno Loureiro +1
This work investigates adversarial training in the context of margin-based linear classifiers in the high-dimensional regime where the dimension and the number of data points $…
Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regression
Lucas Clarté, Adrien Vandenbroucque, Guillaume Dalle +3
We investigate popular resampling methods for estimating the uncertainty of statistical models, such as subsampling, bootstrap and the jackknife, and their performance in high-dime…
A phase transition between positional and semantic learning in a solvable model of dot-product attention
Hugo Cui, Freya Behrens, Florent Krzakala +2
Many empirical studies have provided evidence for the emergence of algorithmic mechanisms (abilities) in the learning of language models, that lead to qualitative improvements of t…