7 papers
Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning
Yatin Dandi, Matteo Vilucchio, Luca Arnaboldi +2
Understanding how deep neural networks learn useful internal representations from data remains a central open problem in the theory of deep learning. We introduce Neural Low-Degree…
On the Existence of Consistent Adversarial Attacks in High-Dimensional Linear Classification
Matteo Vilucchio, Lenka Zdeborová, Bruno Loureiro
What fundamentally distinguishes an adversarial attack from a misclassification due to limited model expressivity or finite data? In this work, we investigate this question in the…
Asymptotics of Non-Convex Generalized Linear Models in High-Dimensions: A proof of the replica formula
Matteo Vilucchio, Yatin Dandi, Matéo Pirio Rossignol +2
The analytic characterization of the high-dimensional behavior of optimization for Generalized Linear Models (GLMs) with Gaussian data has been a central focus in statistics and pr…
On the Geometry of Regularization in Adversarial Training: High-Dimensional Asymptotics and Generalization Bounds
Matteo Vilucchio, Nikolaos Tsilivis, Bruno Loureiro +1
Regularization, whether explicit in terms of a penalty in the loss or implicit in the choice of algorithm, is a cornerstone of modern machine learning. Indeed, controlling the comp…
A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs
Kasimir Tanner, Matteo Vilucchio, Bruno Loureiro +1
This work investigates adversarial training in the context of margin-based linear classifiers in the high-dimensional regime where the dimension and the number of data points $…
Asymptotic Characterisation of Robust Empirical Risk Minimisation Performance in the Presence of Outliers
Matteo Vilucchio, Emanuele Troiani, Vittorio Erba +1
We study robust linear regression in high-dimension, when both the dimension and the number of data points diverge with a fixed ratio , and study a data model that i…
Genealogical Population-Based Training for Hyperparameter Optimization
Antoine Scardigli, Paul Fournier, Matteo Vilucchio +1
HyperParameter Optimization (HPO) aims at finding the best HyperParameters (HPs) of learning models, such as neural networks, in the fastest and most efficient way possible. Most r…