2.5k citations · 3.1k across the 32 of their papers we have counts for
32 papers
Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regime
Hugo Cui, Bruno Loureiro, Florent Krzakala +1
In this manuscript we consider Kernel Ridge Regression (KRR) under the Gaussian design. Exponents for the decay of the excess generalization error of KRR have been reported in vari…
Bayesian reconstruction of memories stored in neural networks from their connectivity
Sebastian Goldt, Florent Krzakala, Lenka Zdeborová +1
The advent of comprehensive synaptic wiring diagrams of large neural circuits has created the field of connectomics and given rise to a number of open research questions. One such…
Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem
Francesca Mignacco, Pierfrancesco Urbani, Lenka Zdeborová
In this paper we investigate how gradient-based algorithms such as gradient descent, (multi-pass) stochastic gradient descent, its persistent variant, and the Langevin algorithm na…
Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed
Maria Refinetti, Sebastian Goldt, Florent Krzakala +1
A recent series of theoretical works showed that the dynamics of neural networks with a certain initialisation are well-captured by kernel methods. Concurrent empirical work demons…
Learning curves of generic features maps for realistic datasets with a teacher-student model
Bruno Loureiro, Cédric Gerbelot, Hugo Cui +4
Teacher-student models provide a framework in which the typical-case performance of high-dimensional supervised learning can be described in closed form. The assumptions of Gaussia…
Solvable Model for Inheriting the Regularization through Knowledge Distillation
Luca Saglietti, Lenka Zdeborová
In recent years the empirical success of transfer learning with neural networks has stimulated an increasing interest in obtaining a theoretical understanding of its core propertie…
Construction of optimal spectral methods in phase retrieval
Antoine Maillard, Florent Krzakala, Yue M. Lu +1
We consider the phase retrieval problem, in which the observer wishes to recover a -dimensional real or complex signal from the (possibly noisy) observation o…
The planted -factor problem
Gabriele Sicuro, Lenka Zdeborová
We consider the problem of recovering an unknown -factor, hidden in a weighted random graph. For this is the planted matching problem, while the case is closely rela…
Epidemic mitigation by statistical inference from contact tracing data
Antoine Baker, Indaco Biazzo, Alfredo Braunstein +10
Contact-tracing is an essential tool in order to mitigate the impact of pandemic such as the COVID-19. In order to achieve efficient and scalable contact-tracing in real time, digi…
Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval
Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota +3
Despite the widespread use of gradient-based algorithms for optimizing high-dimensional non-convex functions, understanding their ability of finding good minima instead of being tr…
Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions
Stefano Sarao Mannelli, Eric Vanden-Eijnden, Lenka Zdeborová
We study the dynamics of optimization and the generalization properties of one-hidden layer neural networks with quadratic activation function in the over-parametrized regime where…
The Gaussian equivalence of generative models for learning with shallow neural networks
Sebastian Goldt, Bruno Loureiro, Galen Reeves +3
Understanding the impact of data structure on the computational tractability of learning is a key challenge for the theory of neural networks. Many theoretical works do not explici…
Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization
Benjamin Aubin, Florent Krzakala, Yue M. Lu +1
We consider a commonly studied supervised classification of a synthetic dataset whose labels are generated by feeding a one-layer neural network with random iid inputs. We study th…
Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification
Francesca Mignacco, Florent Krzakala, Pierfrancesco Urbani +1
We analyze in a closed form the learning dynamics of stochastic gradient descent (SGD) for a single-layer neural network classifying a high-dimensional Gaussian mixture where each…
Phase retrieval in high dimensions: Statistical and computational phase transitions
Antoine Maillard, Bruno Loureiro, Florent Krzakala +1
We consider the phase retrieval problem of reconstructing a -dimensional real or complex signal from (possibly noisy) observations $Y_μ= | \sum_{i=1}^n…
Recovery thresholds in the sparse planted matching problem
Guilhem Semerjian, Gabriele Sicuro, Lenka Zdeborová
We consider the statistical inference problem of recovering an unknown perfect matching, hidden in a weighted random graph, by exploiting the information arising from the use of tw…
The role of regularization in classification of high-dimensional noisy Gaussian mixture
Francesca Mignacco, Florent Krzakala, Yue M. Lu +1
We consider a high-dimensional mixture of two Gaussians in the noisy regime where even an oracle knowing the centers of the clusters misclassifies a small but finite fraction of th…
Generalisation error in learning with random features and the hidden manifold model
Federica Gerace, Bruno Loureiro, Florent Krzakala +2
We study generalised linear regression and classification for a synthetically generated dataset encompassing different problems of interest, such as learning with random features,…
Large deviations for the perceptron model and consequences for active learning
Hugo Cui, Luca Saglietti, Lenka Zdeborová
Active learning is a branch of machine learning that deals with problems where unlabeled data is abundant yet obtaining labels is expensive. The learning algorithm has the possibil…
Rademacher complexity and spin glasses: A link between the replica and statistical theories of learning
Alia Abbara, Benjamin Aubin, Florent Krzakala +1
Statistical learning theory provides bounds of the generalization gap, using in particular the Vapnik-Chervonenkis dimension and the Rademacher complexity. An alternative approach,…
Exact asymptotics for phase retrieval and compressed sensing with random generative priors
Benjamin Aubin, Bruno Loureiro, Antoine Baker +2
We consider the problem of compressed sensing and of (real-valued) phase retrieval with random measurement matrix. We derive sharp asymptotics for the information-theoretically opt…
Machine learning and the physical sciences
Giuseppe Carleo, Ignacio Cirac, Kyle Cranmer +5
Machine learning encompasses a broad range of algorithms and modeling tools used for a vast array of data processing tasks, which has entered most scientific disciplines in recent…
Storage capacity in symmetric binary perceptrons
Benjamin Aubin, Will Perkins, Lenka Zdeborová
We study the problem of determining the capacity of the binary perceptron for two variants of the problem where the corresponding constraint is symmetric. We call these variants th…
Dense Limit of the Dawid-Skene Model for Crowdsourcing and Regions of Sub-optimality of Message Passing Algorithms
Christian Schmidt, Lenka Zdeborová
Crowdsourcing is a strategy to categorize data through the contribution of many individuals. A wide range of theoretical and algorithmic contributions are based on the model of Daw…