149 citations · 455 across the 18 of their papers we have counts for
18 papers
Escape dynamics and implicit bias of one-pass SGD in overparameterized quadratic networks
Dario Bocchi, Theotime Regimbeau, Carlo Lucibello +2
We analyze the one-pass stochastic gradient descent dynamics of a two-layer neural network with quadratic activations in a teacher--student framework. In the high-dimensional regim…
Generative diffusion for perceptron problems: statistical physics analysis and efficient algorithms
Elizaveta Demyanenko, Davide Straziota, Carlo Baldassi +1
We consider random instances of non-convex perceptron problems in the high-dimensional limit of a large number of examples and weights , with finite load . We develo…
The phase diagram of compressed sensing with -norm regularization
Damien Barbier, Carlo Lucibello, Luca Saglietti +2
Noiseless compressive sensing is a two-steps setting that allows for undersampling a sparse signal and then reconstructing it without loss of information. The LASSO algorithm, base…
Random Features Hopfield Networks generalize retrieval to previously unseen examples
Silvio Kalaj, Clarissa Lauditi, Gabriele Perugini +3
It has been recently shown that a learning transition happens when a Hopfield Network stores examples generated as superpositions of random features, where new attractors correspon…
The star-shaped space of solutions of the spherical negative perceptron
Brandon Livio Annesi, Clarissa Lauditi, Carlo Lucibello +4
Empirical studies on the landscape of neural networks have shown that low-energy configurations are often found in complex connected structures, where zero-energy paths between pai…
Storage and Learning phase transitions in the Random-Features Hopfield Model
Matteo Negri, Clarissa Lauditi, Gabriele Perugini +2
The Hopfield model is a paradigmatic model of neural networks that has been analyzed for many decades in the statistical physics, neuroscience, and machine learning communities. In…
Compressed sensing with l0-norm: statistical physics analysis and algorithms for signal recovery
D. Barbier, C Lucibello, L. Saglietti +2
Noiseless compressive sensing is a protocol that enables undersampling and later recovery of a signal without loss of information. This compression is possible because the signal i…
Neural networks: from the perceptron to deep nets
Marylou Gabrié, Surya Ganguli, Carlo Lucibello +1
Artificial networks have been studied through the prism of statistical mechanics as disordered systems since the 80s, starting from the simple models of Hopfield's associative memo…
Deep learning via message passing algorithms based on belief propagation
Carlo Lucibello, Fabrizio Pittorino, Gabriele Perugini +1
Message-passing algorithms based on the Belief Propagation (BP) equations constitute a well-known distributed computational scheme. It is exact on tree-like graphical models and ha…
Entropic gradient descent algorithms and wide flat minima
Fabrizio Pittorino, Carlo Lucibello, Christoph Feinauer +4
The properties of flat minima in the empirical risk landscape of neural networks have been debated for some time. Increasing evidence suggests they possess better generalization ca…
Clustering of solutions in the symmetric binary perceptron
Carlo Baldassi, Riccardo Della Vecchia, Carlo Lucibello +1
The geometrical features of the (non-convex) loss landscape of neural network models are crucial in ensuring successful optimization and, most importantly, the capability to genera…
Generalized Approximate Survey Propagation for High-Dimensional Estimation
Luca Saglietti, Yue M. Lu, Carlo Lucibello
In Generalized Linear Estimation (GLE) problems, we seek to estimate a signal that is observed through a linear transform followed by a component-wise, possibly nonlinear and noisy…
Critical initialisation in continuous approximations of binary neural networks
George Stamatescu, Federica Gerace, Carlo Lucibello +2
The training of stochastic neural network models with binary () weights and activations via continuous surrogate networks is investigated. We derive new surrogates using a no…
On the role of synaptic stochasticity in training low-precision neural networks
Carlo Baldassi, Federica Gerace, Hilbert J. Kappen +4
Stochasticity and limited precision of synaptic weights in neural network models are key aspects of both biological and hardware modeling of learning processes. Here we show that a…
Unreasonable Effectiveness of Learning Neural Networks: From Accessible States and Robust Ensembles to Basic Algorithmic Schemes
Carlo Baldassi, Christian Borgs, Jennifer Chayes +4
In artificial neural networks, learning from data is a computationally demanding task in which a large number of connection weights are iteratively tuned through stochastic-gradien…
Learning may need only a few bits of synaptic precision
Carlo Baldassi, Federica Gerace, Carlo Lucibello +2
Learning in neural networks poses peculiar challenges when using discretized rather then continuous synaptic states. The choice of discrete synapses is motivated by biological reas…
Local entropy as a measure for sampling solutions in Constraint Satisfaction Problems
Carlo Baldassi, Alessandro Ingrosso, Carlo Lucibello +2
We introduce a novel Entropy-driven Monte Carlo (EdMC) strategy to efficiently sample solutions of random Constraint Satisfaction Problems (CSPs). First, we extend a recent result…
Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses
Carlo Baldassi, Alessandro Ingrosso, Carlo Lucibello +2
We show that discrete synaptic weights can be efficiently used for learning in large scale neural systems, and lead to unanticipated computational performance. We focus on the repr…