82 citations · 360 across the 42 of their papers we have counts for
8 papers · 1 filter
A Random Matrix Theory Perspective on the Spectrum of Learned Features and Asymptotic Generalization Capabilities
Yatin Dandi, Luca Pesce, Hugo Cui +3
A key property of neural networks is their capacity of adapting to data during training. Yet, our current mathematical understanding of feature learning and its relationship to gen…
Online Learning and Information Exponents: On The Importance of Batch size, and Time/Complexity Tradeoffs
Luca Arnaboldi, Yatin Dandi, Florent Krzakala +3
We study the impact of the batch size on the iteration time of training two-layer neural networks with one-pass stochastic gradient descent (SGD) on multi-index target fu…
Dimension-free deterministic equivalents and scaling laws for random feature regression
Leonardo Defilippis, Bruno Loureiro, Theodor Misiakiewicz
In this work we investigate the generalization performance of random feature ridge regression (RFRR). Our main contribution is a general deterministic equivalent for the test error…
Fundamental computational limits of weak learnability in high-dimensional multi-index models
Emanuele Troiani, Yatin Dandi, Leonardo Defilippis +3
Multi-index models - functions which only depend on the covariates through a non-linear transformation of their projection on a subspace - are a useful benchmark for investigating…
Asymptotics of Learning with Deep Structured (Random) Features
Dominik Schröder, Daniil Dmitriev, Hugo Cui +1
For a large class of feature maps we provide a tight asymptotic characterisation of the test error associated with learning the readout layer, in the high-dimensional limit where t…
Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regression
Lucas Clarté, Adrien Vandenbroucque, Guillaume Dalle +3
We investigate popular resampling methods for estimating the uncertainty of statistical models, such as subsampling, bootstrap and the jackknife, and their performance in high-dime…
A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs
Kasimir Tanner, Matteo Vilucchio, Bruno Loureiro +1
This work investigates adversarial training in the context of margin-based linear classifiers in the high-dimensional regime where the dimension and the number of data points $…
Asymptotics of feature learning in two-layer networks after one gradient-step
Hugo Cui, Luca Pesce, Yatin Dandi +4
In this manuscript, we investigate the problem of how two-layer neural networks learn features from data, and improve over the kernel regime, after being trained with a single grad…