23 citations · 89 across the 18 of their papers we have counts for
19 papers
Information-Theoretic Limits for Rectangular Spiked Matrix Models with Rotationally Invariant Noise
Qun Chen, Rudy Skerk, Jean Barbier +3
We study the information-theoretic limits of inference in rectangular rank-one spiked matrix models with additive rotationally invariant noise, thereby going beyond the standard as…
Variational Bounds for Perceptron Learning from Structured Data
Francesco Camilli, Pierluigi Contucci, Federica Gerace +1
We introduce a variational approach to a finite-temperature continuous-spin perceptron trained on a Gaussian mixture. The model allows for a broad class of concave utilities and lo…
From entropic constraints to reinforced processes: a probabilistic origin of multiscale measures
Francesco Camilli, Pierluigi Contucci, Emanuele Mingione
We study multiscale measures, defined by a hierarchy of partial Gibbs equilibria, from variational and probabilistic perspectives. We first show that they are the unique solutions…
Statistical physics of deep learning: Optimal learning of a multi-layer perceptron near interpolation
Jean Barbier, Francesco Camilli, Minh-Toan Nguyen +2
For four decades statistical physics has been providing a framework to analyse neural networks. A long-standing question remained on its capacity to tackle deep learning models cap…
Information-theoretic reduction of deep neural networks to linear models in the overparametrized proportional regime
Francesco Camilli, Daria Tieplova, Eleonora Bergamin +1
We rigorously analyse fully-trained neural networks of arbitrary depth in the Bayesian optimal setting in the so-called proportional scaling regime where the number of training sam…
Statistical mechanics of extensive-width Bayesian neural networks near interpolation
Jean Barbier, Francesco Camilli, Minh-Toan Nguyen +2
For three decades statistical mechanics has been providing a framework to analyse neural networks. However, the theoretically tractable models, e.g., perceptrons, random features m…
Limit theorems for the non-convex multispecies Curie-Weiss model
Francesco Camilli, Emanuele Mingione, Godwin Osabutey
We study the thermodynamic properties of the generalized non-convex multispecies Curie-Weiss model, where interactions among different types of particles (forming the species) are…
Optimal generalisation and learning transition in extensive-width shallow neural networks near interpolation
Jean Barbier, Francesco Camilli, Minh-Toan Nguyen +2
We consider a teacher-student model of supervised learning with a fully-trained two-layer neural network whose width and input dimension are large and proportional. We prov…
On the phase diagram of the multiscale mean-field spin-glass
Francesco Camilli, Pierluigi Contucci, Emanuele Mingione +1
In this paper we study the phase diagram of a Sherrington-Kirkpatrick (SK) model where the couplings are forced to thermalize at different time scales. Besides being a challenging…
On the phase diagram of extensive-rank symmetric matrix denoising beyond rotational invariance
Jean Barbier, Francesco Camilli, Justin Ko +1
Matrix denoising is central to signal processing and machine learning. Its statistical analysis when the matrix to infer has a factorised structure with a rank growing proportional…
Information limits and Thouless-Anderson-Palmer equations for spiked matrix models with structured noise
Jean Barbier, Francesco Camilli, Marco Mondelli +1
We consider a prototypical problem of Bayesian inference for a structured spiked model: a low-rank signal is corrupted by additive noise. While both information-theoretic and algor…
The Decimation Scheme for Symmetric Matrix Factorization
Francesco Camilli, Marc Mézard
Matrix factorization is an inference problem that has acquired importance due to its vast range of applications that go from dictionary learning to recommendation systems and machi…
Fundamental limits of overparametrized shallow neural networks for supervised learning
Francesco Camilli, Daria Tieplova, Jean Barbier
We carry out an information-theoretical analysis of a two-layer neural network trained from input-output pairs generated by a teacher network with matching architecture, in overpar…
Central limit theorem for the overlaps on the Nishimori line
Francesco Camilli, Pierluigi Contucci, Emanuele Mingione
The overlap distribution of the Sherrington-Kirkpatrick model on the Nishimori line has been proved to be self averaging for large volumes. Here we study the joint distribution of…
Matrix factorization with neural networks
Francesco Camilli, Marc Mézard
Matrix factorization is an important mathematical problem encountered in the context of dictionary learning, recommendation systems and machine learning. We introduce a new `decima…
Bayes-optimal limits in structured PCA, and how to reach them
Jean Barbier, Francesco Camilli, Marco Mondelli +1
How do statistical dependencies in measurement noise influence high-dimensional inference? To answer this, we study the paradigmatic spiked matrix model of principal components ana…
An inference problem in a mismatched setting: a spin-glass model with Mattis interaction
Francesco Camilli, Pierluigi Contucci, Emanuele Mingione
The Wigner spiked model in a mismatched setting is studied with the finite temperature Statistical Mechanics approach through its representation as a Sherrington-Kirkpatrick model…
The solution of the deep Boltzmann machine on the Nishimori line
Diego Alberici, Francesco Camilli, Pierluigi Contucci +1
The deep Boltzmann machine on the Nishimori line with a finite number of layers is exactly solved by a theorem that expresses its pressure through a finite dimensional variational…
The multi-species mean-field spin-glass on the Nishimori line
Diego Alberici, Francesco Camilli, Pierluigi Contucci +1
In this paper we study a multi-species disordered model on the Nishimori line. The typical properties of this line, a set of identities and inequalities among correlation functions…