6 papers
Statistical physics of deep learning: Optimal learning of a multi-layer perceptron near interpolation
Jean Barbier, Francesco Camilli, Minh-Toan Nguyen +2
For four decades statistical physics has been providing a framework to analyse neural networks. A long-standing question remained on its capacity to tackle deep learning models cap…
Feature learning in finite-width Bayesian deep linear networks with multiple outputs and convolutional layers
Federico Bassetti, Marco Gherardi, Alessandro Ingrosso +2
Deep linear networks have been extensively studied, as they provide simplified models of deep learning. However, little is known in the case of finite-width architectures with mult…
Statistical mechanics of extensive-width Bayesian neural networks near interpolation
Jean Barbier, Francesco Camilli, Minh-Toan Nguyen +2
For three decades statistical mechanics has been providing a framework to analyse neural networks. However, the theoretically tractable models, e.g., perceptrons, random features m…
Optimal generalisation and learning transition in extensive-width shallow neural networks near interpolation
Jean Barbier, Francesco Camilli, Minh-Toan Nguyen +2
We consider a teacher-student model of supervised learning with a fully-trained two-layer neural network whose width and input dimension are large and proportional. We prov…
Restoring balance: principled under/oversampling of data for optimal classification
Emanuele Loffredo, Mauro Pastore, Simona Cocco +1
Class imbalance in real-world data poses a common bottleneck for machine learning tasks, since achieving good generalization on under-represented examples is often challenging. Mit…
Random features and polynomial rules
Fabián Aguirre-López, Silvio Franz, Mauro Pastore
Random features models play a distinguished role in the theory of deep learning, describing the behavior of neural networks close to their infinite-width limit. In this work, we pr…