7 papers
Learning and Testing Inverse Statistical Problems For Interacting Systems Undergoing Phase Transition
Stefano Bae, Dario Bocchi, Luca Maria Del Bono +1
Inverse problems arise in situations where data is available, but the underlying model is not. It can therefore be necessary to infer the parameters of the latter starting from the…
Spectral phase transitions and trainability in neural network learning dynamics
Chanju Park, Dario Bocchi, Francesco D'Amico +2
The emergence of low-dimensional structures in the spectra of neural network weight matrices is a common empirical feature of trained models, but the dynamical origin of this pheno…
Discontinuous BBP transitions
Dario Bocchi, Giulio Biroli, Chiara Cammarota +1
The Baik-Ben Arous-Peche (BBP) transition sets fundamental limits for detecting low-rank structure in noisy high-dimensional data and underlies a wide range of spectral methods in…
Implicit bias produces neural scaling laws in learning curves, from perceptrons to deep networks
Francesco D'Amico, Dario Bocchi, Matteo Negri
Scaling laws in deep learning -- empirical power-law relationships linking model performance to resource growth -- have emerged as simple yet striking regularities across architect…
Escape dynamics and implicit bias of one-pass SGD in overparameterized quadratic networks
Dario Bocchi, Theotime Regimbeau, Carlo Lucibello +2
We analyze the one-pass stochastic gradient descent dynamics of a two-layer neural network with quadratic activations in a teacher--student framework. In the high-dimensional regim…
Overparametrization bends the landscape: BBP transitions at initialization in simple Neural Networks
Brandon Livio Annesi, Dario Bocchi, Chiara Cammarota
High-dimensional non-convex loss landscapes play a central role in the theory of Machine Learning. Gaining insight into how these landscapes interact with gradient-based optimizati…