75 citations · 253 across the 9 of their papers we have counts for
10 papers
Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup
Sebastian Goldt, Madhu S. Advani, Andrew M. Saxe +2
Deep neural networks achieve stellar generalisation even when they have enough parameters to easily fit all their training data. We study this phenomenon by analysing the dynamics…
Dimensionality compression and expansion in Deep Neural Networks
Stefano Recanatesi, Matthew Farrell, Madhu Advani +3
Datasets such as images, text, or movies are embedded in high-dimensional spaces. However, in important cases such as images of objects, the statistical structure in the data const…
Generalisation dynamics of online learning in over-parameterised neural networks
Sebastian Goldt, Madhu S. Advani, Andrew M. Saxe +2
Deep neural networks achieve stellar generalisation on a variety of problems, despite often being large enough to easily fit all their training data. Here we study the generalisati…
Minnorm training: an algorithm for training over-parameterized deep neural networks
Yamini Bansal, Madhu Advani, David D Cox +1
In this work, we propose a new training method for finding minimum weight norm solutions in over-parameterized neural networks (NNs). This method seeks to improve training speed an…
Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learning
Yao Zhang, Andrew M. Saxe, Madhu S. Advani +1
Finding parameters that minimise a loss function is at the core of many machine learning methods. The Stochastic Gradient Descent algorithm is widely used and delivers state of the…
High-dimensional dynamics of generalization error in neural networks
Madhu S. Advani, Andrew M. Saxe
We perform an average case analysis of the generalization dynamics of large neural networks trained using gradient descent. We study the practically-relevant "high-dimensional" reg…
Environmental engineering is an emergent feature of diverse ecosystems and drives community structure
Madhu Advani, Guy Bunin, Pankaj Mehta
A central question in ecology is to understand the ecological processes that shape community structure. Niche-based theories have emphasized the important role played by competitio…
An equivalence between high dimensional Bayes optimal inference and M-estimation
Madhu Advani, Surya Ganguli
When recovering an unknown signal from noisy measurements, the computational difficulty of performing optimal Bayesian MMSE (minimum mean squared error) inference often necessitate…
Statistical Mechanics of High-Dimensional Inference
Madhu Advani, Surya Ganguli
To model modern large-scale datasets, we need efficient algorithms to infer a set of unknown model parameters from noisy measurements. What are fundamental limits on the ac…
Statistical mechanics of complex neural systems and high dimensional data
Madhu Advani, Subhaneil Lahiri, Surya Ganguli
Recent experimental advances in neuroscience have opened new vistas into the immense complexity of neuronal networks. This proliferation of data challenges us on two parallel front…