30 citations · 112 across the 22 of their papers we have counts for
22 papers
How Width and Data Shape Generalization Scaling Laws in Quadratic Neural Networks
Julius Girardin, Emanuele Troiani, Yizhou Xu +3
Understanding how performance scales jointly with model size and data is a central problem in modern machine learning. Existing theoretical works on scaling laws typically describe…
Percolation and clustering in ecological communities: A dynamical theory
Dario Sergo, Cédric Koller, Vittorio Erba +1
Ecological communities with structured interactions exhibit collective phenomena such as percolation and clustering of occupied sites. While these effects have been documented in e…
A solvable high-dimensional model where nonlinear autoencoders learn structure invisible to PCA while test loss misaligns with generalization
Vicente Conde Mendes, Lorenzo Bardone, Cédric Koller +4
Many real-world datasets contain hidden structure that cannot be detected by simple linear correlations between input features. For example, latent factors may influence the data i…
Single-Head Attention in High Dimensions: A Theory of Generalization, Weights Spectra, and Scaling Laws
Fabrizio Boncoraglio, Vittorio Erba, Emanuele Troiani +3
Trained attention layers exhibit striking and reproducible spectral structure of the weights, including low-rank collapse, bulk deformation, and isolated spectral outliers, yet the…
Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regime
Leonardo Defilippis, Yizhou Xu, Julius Girardin +5
Neural scaling laws underlie many of the recent advances in deep learning, yet their theoretical understanding remains largely confined to linear models. In this work, we present a…
The maximum-average subtensor problem: equilibrium and out-of-equilibrium properties
Vittorio Erba, Nathan Malo Kupferschmid, Rodrigo Pérez Ortiz +1
In this paper we introduce and study the Maximum-Average Subtensor (-MAS) problem, in which one wants to find a subtensor of size of a given random tensor of size , both…
Bayes optimal learning of attention-indexed models
Fabrizio Boncoraglio, Emanuele Troiani, Vittorio Erba +1
We introduce the attention-indexed model (AIM), a theoretical framework for analyzing learning in deep attention layers. Inspired by multi-index models, AIM captures how token-leve…
The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networks
Vittorio Erba, Emanuele Troiani, Lenka Zdeborová +1
We study the high-dimensional asymptotics of empirical risk minimization (ERM) in over-parametrized two-layer neural networks with quadratic activations trained on synthetic data.…
Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-dimensional Tokens
Vittorio Erba, Emanuele Troiani, Luca Biggio +2
Current progress in artificial intelligence is centered around so-called large language models that consist of neural networks processing long sequences of high-dimensional vectors…
Quenches in the Sherrington-Kirkpatrick model
Vittorio Erba, Freya Behrens, Florent Krzakala +1
The Sherrington-Kirkpatrick (SK) model is a prototype of a complex non-convex energy landscape. Dynamical processes evolving on such landscapes and locally aiming to reach minima a…
Les Houches Lectures on Deep Learning at Large & Infinite Width
Yasaman Bahri, Boris Hanin, Antonin Brossollet +4
These lectures, presented at the 2022 Les Houches Summer School on Statistical Physics and Machine Learning, focus on the infinite-width limit and large-width regime of deep neural…
Asymptotic Characterisation of Robust Empirical Risk Minimisation Performance in the Presence of Outliers
Matteo Vilucchio, Emanuele Troiani, Vittorio Erba +1
We study robust linear regression in high-dimension, when both the dimension and the number of data points diverge with a fixed ratio , and study a data model that i…
Statistical mechanics of the maximum-average submatrix problem
Vittorio Erba, Florent Krzakala, Rodrigo Pérez +1
We study the maximum-average submatrix problem, in which given an matrix one needs to find the submatrix with the largest average of entries. We study…
Optimal denoising of rotationally invariant rectangular matrices
Emanuele Troiani, Vittorio Erba, Florent Krzakala +2
In this manuscript we consider denoising of large rectangular matrices: given a noisy observation of a signal matrix, what is the best way of recovering the signal matrix itself? F…
The number of optimal matchings for Euclidean Assignment on the line
Sergio Caracciolo, Vittorio Erba, Andrea Sportiello
We consider the Random Euclidean Assignment Problem in dimension , with linear cost function. In this version of the problem, in general, there is a large degeneracy of the gr…
Self-induced glassy phase in multimodal cavity quantum electrodynamics
Vittorio Erba, Mauro Pastore, Pietro Rotondo
We provide strong evidence that the effective spin-spin interaction in a multimodal confocal optical cavity gives rise to a self-induced glassy phase, which emerges exclusively fro…
The p-Airy distribution
Sergio Caracciolo, Vittorio Erba, Andrea Sportiello
In this manuscript we consider the set of Dyck paths equipped with the uniform measure, and we study the statistical properties of a deformation of the observable "area below the D…
Statistical learning theory of structured data
Mauro Pastore, Pietro Rotondo, Vittorio Erba +1
The traditional approach of statistical physics to supervised learning routinely assumes unrealistic generative models for the data: usually inputs are independent random variables…
Random geometric graphs in high dimension
Vittorio Erba, Sebastiano Ariosto, Marco Gherardi +1
Many machine learning algorithms used for dimensional reduction and manifold learning leverage on the computation of the nearest neighbours to each point of a dataset to perform th…
Intrinsic dimension estimation for locally undersampled data
Vittorio Erba, Marco Gherardi, Pietro Rotondo
High-dimensional data are ubiquitous in contemporary science and finding methods to compress them is one of the primary goals of machine learning. Given a dataset lying in a high-d…
The Dyck bound in the concave 1-dimensional random assignment model
Sergio Caracciolo, Matteo P. D'Achille, Vittorio Erba +1
We consider models of assignment for random blue points and red points on an interval of length , in which the cost for connecting a blue point in to a red point in…
Unified Fock space representation of fractional quantum Hall states
Andrea Di Gioacchino, Luca Guido Molinari, Vittorio Erba +1
Many bosonic (fermionic) fractional quantum Hall states, such as Laughlin, Moore-Read and Read-Rezayi wavefunctions, belong to a special class of orthogonal polynomials: the Jack p…