486 citations · 2.8k across the 59 of their papers we have counts for
6 papers · 1 filter
Generative modeling through internal high-dimensional chaotic activity
Samantha J. Fournier, Pierfrancesco Urbani
Generative modeling aims at producing new datapoints whose statistical properties resemble the ones in a training dataset. In recent years, there has been a burst of machine learni…
Stochastic Gradient Descent outperforms Gradient Descent in recovering a high-dimensional signal in a glassy energy landscape
Persia Jana Kamali, Pierfrancesco Urbani
Stochastic Gradient Descent (SGD) is an out-of-equilibrium algorithm used extensively to train artificial neural networks. However very little is known on to what extent SGD is cru…
Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval
Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota +3
Despite the widespread use of gradient-based algorithms for optimizing high-dimensional non-convex functions, understanding their ability of finding good minima instead of being tr…
Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification
Francesca Mignacco, Florent Krzakala, Pierfrancesco Urbani +1
We analyze in a closed form the learning dynamics of stochastic gradient descent (SGD) for a single-layer neural network classifying a high-dimensional Gaussian mixture where each…
Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Models
Stefano Sarao Mannelli, Florent Krzakala, Pierfrancesco Urbani +1
In this work we analyse quantitatively the interplay between the loss landscape and performance of descent algorithms in a prototypical inference problem, the spiked matrix-tensor…
Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference
Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota +3
Gradient-descent-based algorithms and their stochastic versions have widespread applications in machine learning and statistical inference. In this work we perform an analytic stud…