149 citations · 453 across the 21 of their papers we have counts for
21 papers
Escape dynamics and implicit bias of one-pass SGD in overparameterized quadratic networks
Dario Bocchi, Theotime Regimbeau, Carlo Lucibello +2
We analyze the one-pass stochastic gradient descent dynamics of a two-layer neural network with quadratic activations in a teacher--student framework. In the high-dimensional regim…
The phase diagram of compressed sensing with -norm regularization
Damien Barbier, Carlo Lucibello, Luca Saglietti +2
Noiseless compressive sensing is a two-steps setting that allows for undersampling a sparse signal and then reconstructing it without loss of information. The LASSO algorithm, base…
Tilting the Odds at the Lottery: the Interplay of Overparameterisation and Curricula in Neural Networks
Stefano Sarao Mannelli, Yaraslau Ivashynka, Andrew Saxe +1
A wide range of empirical and theoretical works have shown that overparameterisation can amplify the performance of neural networks. According to the lottery ticket hypothesis, ove…
The twin peaks of learning neural networks
Elizaveta Demyanenko, Christoph Feinauer, Enrico M. Malatesta +1
Recent works demonstrated the existence of a double-descent phenomenon for the generalization error of neural networks, where highly overparameterized models escape overfitting and…
The star-shaped space of solutions of the spherical negative perceptron
Brandon Livio Annesi, Clarissa Lauditi, Carlo Lucibello +4
Empirical studies on the landscape of neural networks have shown that low-energy configurations are often found in complex connected structures, where zero-energy paths between pai…
Compressed sensing with l0-norm: statistical physics analysis and algorithms for signal recovery
D. Barbier, C Lucibello, L. Saglietti +2
Noiseless compressive sensing is a protocol that enables undersampling and later recovery of a signal without loss of information. This compression is possible because the signal i…
Optimal transfer protocol by incremental layer defrosting
Federica Gerace, Diego Doimo, Stefano Sarao Mannelli +2
Transfer learning is a powerful tool enabling model training with limited amounts of data. This technique is particularly useful in real-world problems where data availability is o…
Bias-inducing geometries: an exactly solvable data model with fairness implications
Stefano Sarao Mannelli, Federica Gerace, Negar Rostamzadeh +1
Machine learning (ML) may be oblivious to human bias but it is not immune to its perpetuation. Marginalisation and iniquitous group representation are often traceable in the very d…
Large Deviations of Semi-supervised Learning in the Stochastic Block Model
Hugo Cui, Luca Saglietti, Lenka Zdeborová
In community detection on graphs, the semi-supervised learning problem entails inferring the ground-truth membership of each node in a graph, given the connectivity structure and a…
An Analytical Theory of Curriculum Learning in Teacher-Student Networks
Luca Saglietti, Stefano Sarao Mannelli, Andrew Saxe
In humans and animals, curriculum learning -- presenting data in a curated order - is critical to rapid learning and effective pedagogy. Yet in machine learning, curricula are not…
Probing transfer learning with a model of synthetic correlated datasets
Federica Gerace, Luca Saglietti, Stefano Sarao Mannelli +2
Transfer learning can significantly improve the sample efficiency of neural networks, by exploiting the relatedness between a data-scarce target task and a data-abundant source tas…
Solvable Model for Inheriting the Regularization through Knowledge Distillation
Luca Saglietti, Lenka Zdeborová
In recent years the empirical success of transfer learning with neural networks has stimulated an increasing interest in obtaining a theoretical understanding of its core propertie…
Large deviations for the perceptron model and consequences for active learning
Hugo Cui, Luca Saglietti, Lenka Zdeborová
Active learning is a branch of machine learning that deals with problems where unlabeled data is abundant yet obtaining labels is expensive. The learning algorithm has the possibil…
Generalized Approximate Survey Propagation for High-Dimensional Estimation
Luca Saglietti, Yue M. Lu, Carlo Lucibello
In Generalized Linear Estimation (GLE) problems, we seek to estimate a signal that is observed through a linear transform followed by a component-wise, possibly nonlinear and noisy…
Gaussian Process Prior Variational Autoencoders
Francesco Paolo Casale, Adrian V Dalca, Luca Saglietti +2
Variational autoencoders (VAE) are a powerful and widely-used class of models to learn complex data distributions in an unsupervised fashion. One important limitation of VAEs is th…
From statistical inference to a differential learning rule for stochastic neural networks
Luca Saglietti, Federica Gerace, Alessandro Ingrosso +2
Stochastic neural networks are a prototypical computational device able to build a probabilistic representation of an ensemble of external stimuli. Building on the relationship bet…
On the role of synaptic stochasticity in training low-precision neural networks
Carlo Baldassi, Federica Gerace, Hilbert J. Kappen +4
Stochasticity and limited precision of synaptic weights in neural network models are key aspects of both biological and hardware modeling of learning processes. Here we show that a…
Unreasonable Effectiveness of Learning Neural Networks: From Accessible States and Robust Ensembles to Basic Algorithmic Schemes
Carlo Baldassi, Christian Borgs, Jennifer Chayes +4
In artificial neural networks, learning from data is a computationally demanding task in which a large number of connection weights are iteratively tuned through stochastic-gradien…
Learning may need only a few bits of synaptic precision
Carlo Baldassi, Federica Gerace, Carlo Lucibello +2
Learning in neural networks poses peculiar challenges when using discretized rather then continuous synaptic states. The choice of discrete synapses is motivated by biological reas…
Local entropy as a measure for sampling solutions in Constraint Satisfaction Problems
Carlo Baldassi, Alessandro Ingrosso, Carlo Lucibello +2
We introduce a novel Entropy-driven Monte Carlo (EdMC) strategy to efficiently sample solutions of random Constraint Satisfaction Problems (CSPs). First, we extend a recent result…
Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses
Carlo Baldassi, Alessandro Ingrosso, Carlo Lucibello +2
We show that discrete synaptic weights can be efficiently used for learning in large scale neural systems, and lead to unanticipated computational performance. We focus on the repr…