8 citations · 24 across the 6 of their papers we have counts for
4 papers · 1 filter
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity
Sebastian Lee, Sebastian Goldt, Andrew Saxe
Continual learning-the ability to learn many tasks in sequence-is critical for artificial learning systems. Yet standard training methods for deep networks often suffer from catast…
Generalisation dynamics of online learning in over-parameterised neural networks
Sebastian Goldt, Madhu S. Advani, Andrew M. Saxe +2
Deep neural networks achieve stellar generalisation on a variety of problems, despite often being large enough to easily fit all their training data. Here we study the generalisati…
Minnorm training: an algorithm for training over-parameterized deep neural networks
Yamini Bansal, Madhu Advani, David D Cox +1
In this work, we propose a new training method for finding minimum weight norm solutions in over-parameterized neural networks (NNs). This method seeks to improve training speed an…
High-dimensional dynamics of generalization error in neural networks
Madhu S. Advani, Andrew M. Saxe
We perform an average case analysis of the generalization dynamics of large neural networks trained using gradient descent. We study the practically-relevant "high-dimensional" reg…