35 citations · 68 across the 11 of their papers we have counts for
12 papers
A Solvable Model of Adaptive Learning Rate Rescaling: Acceleration, Stability & Scaling
Itay Lavie, Clarissa Lauditi, Cengiz Pehlevan
A recurring design principle in modern optimizers is to decouple update magnitude from the raw gradient norm, yet its consequences for learning-curve and resource scaling remain un…
Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer
Clarissa Lauditi, Cengiz Pehlevan, Blake Bordelon
We study the evolution of hidden-weight spectra in wide neural networks trained by (stochastic) gradient descent. We develop a two-level dynamical mean-field theory (DMFT) that joi…
Transfer Learning in Infinite Width Feature Learning Networks
Clarissa Lauditi, Blake Bordelon, Cengiz Pehlevan
We develop a theory of transfer learning in infinitely wide neural networks under gradient flow that quantifies when pretraining on a source task improves generalization on a targe…
Generalization performance of narrow one-hidden layer networks in the teacher-student setting
Rodrigo Pérez Ortiz, Gibbs Nwemadji, Jean Barbier +4
Understanding the generalization properties of neural networks on simple input-output distributions is key to explaining their performance on real datasets. The classical teacher-s…
Adaptive kernel predictors from feature-learning infinite limits of neural networks
Clarissa Lauditi, Blake Bordelon, Cengiz Pehlevan
Previous influential work showed that infinite width limits of neural networks in the lazy training regime are described by kernel machines. Here, we show that neural networks trai…
Random Features Hopfield Networks generalize retrieval to previously unseen examples
Silvio Kalaj, Clarissa Lauditi, Gabriele Perugini +3
It has been recently shown that a learning transition happens when a Hopfield Network stores examples generated as superpositions of random features, where new attractors correspon…
Impact of dendritic non-linearities on the computational capabilities of neurons
Clarissa Lauditi, Enrico M. Malatesta, Fabrizio Pittorino +3
How neurons integrate the myriad synaptic inputs scattered across their dendrites is a fundamental question in neuroscience. Multiple neurophysiological experiments have shown that…
Sparse Representations, Inference and Learning
Clarissa Lauditi, Emanuele Troiani, Marc Mézard
In recent years statistical physics has proven to be a valuable tool to probe into large dimensional inference problems such as the ones occurring in machine learning. Statistical…
The star-shaped space of solutions of the spherical negative perceptron
Brandon Livio Annesi, Clarissa Lauditi, Carlo Lucibello +4
Empirical studies on the landscape of neural networks have shown that low-energy configurations are often found in complex connected structures, where zero-energy paths between pai…
Storage and Learning phase transitions in the Random-Features Hopfield Model
Matteo Negri, Clarissa Lauditi, Gabriele Perugini +3
The Hopfield model is a paradigmatic model of neural networks that has been analyzed for many decades in the statistical physics, neuroscience, and machine learning communities. In…
Learning through atypical "phase transitions" in overparameterized neural networks
Carlo Baldassi, Clarissa Lauditi, Enrico M. Malatesta +3
Current deep neural networks are highly overparameterized (up to billions of connection weights) and nonlinear. Yet they can fit data almost perfectly through variants of gradient…
Unveiling the structure of wide flat minima in neural networks
Carlo Baldassi, Clarissa Lauditi, Enrico M. Malatesta +2
The success of deep learning has revealed the application potential of neural networks across the sciences and opened up fundamental theoretical problems. In particular, the fact t…