75 citations · 253 across the 9 of their papers we have counts for
6 papers · 1 filter
Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup
Sebastian Goldt, Madhu S. Advani, Andrew M. Saxe +2
Deep neural networks achieve stellar generalisation even when they have enough parameters to easily fit all their training data. We study this phenomenon by analysing the dynamics…
Generalisation dynamics of online learning in over-parameterised neural networks
Sebastian Goldt, Madhu S. Advani, Andrew M. Saxe +2
Deep neural networks achieve stellar generalisation on a variety of problems, despite often being large enough to easily fit all their training data. Here we study the generalisati…
Minnorm training: an algorithm for training over-parameterized deep neural networks
Yamini Bansal, Madhu Advani, David D Cox +1
In this work, we propose a new training method for finding minimum weight norm solutions in over-parameterized neural networks (NNs). This method seeks to improve training speed an…
High-dimensional dynamics of generalization error in neural networks
Madhu S. Advani, Andrew M. Saxe
We perform an average case analysis of the generalization dynamics of large neural networks trained using gradient descent. We study the practically-relevant "high-dimensional" reg…
An equivalence between high dimensional Bayes optimal inference and M-estimation
Madhu Advani, Surya Ganguli
When recovering an unknown signal from noisy measurements, the computational difficulty of performing optimal Bayesian MMSE (minimum mean squared error) inference often necessitate…
Statistical Mechanics of High-Dimensional Inference
Madhu Advani, Surya Ganguli
To model modern large-scale datasets, we need efficient algorithms to infer a set of unknown model parameters from noisy measurements. What are fundamental limits on the ac…