6 papers
Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrieval
Antoine Maillard, Tony Bonnaire, Giulio Biroli
We consider the landscape of empirical risk minimization for high-dimensional Gaussian single-index models (generalized linear models). The objective is to recover an unknown signa…
Optimal scaling laws in learning hierarchical multi-index models
Leonardo Defilippis, Florent Krzakala, Bruno Loureiro +1
In this work, we provide a sharp theory of scaling laws for two-layer neural networks trained on a class of hierarchical multi-index targets, in a genuinely representation-limited…
Fundamental Limits of Matrix Sensing: Exact Asymptotics, Universality, and Applications
Yizhou Xu, Antoine Maillard, Lenka Zdeborová +1
In the matrix sensing problem, one wishes to reconstruct a matrix from (possibly noisy) observations of its linear projections along given directions. We consider this model in the…
Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-dimensional Tokens
Vittorio Erba, Emanuele Troiani, Luca Biggio +2
Current progress in artificial intelligence is centered around so-called large language models that consist of neural networks processing long sequences of high-dimensional vectors…
Injectivity of ReLU networks: perspectives from statistical physics
Antoine Maillard, Afonso S. Bandeira, David Belius +2
When can the input of a ReLU neural network be inferred from its output? In other words, when is the network injective? We consider a single layer, , w…
Fitting an ellipsoid to a quadratic number of random points
Afonso S. Bandeira, Antoine Maillard, Shahar Mendelson +1
We consider the problem of fitting standard Gaussian random vectors in to the boundary of a centered ellipsoid, as . This problem…