1.6k citations · 3.9k across the 64 of their papers we have counts for
12 papers · 1 filter
Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features
Adityanarayanan Radhakrishnan, Daniel Beaglehole, Parthe Pandit +1
In recent years neural networks have achieved impressive results on many technological and scientific tasks. Yet, the mechanism through which these models automatically select feat…
Restricted Strong Convexity of Deep Learning Models with Smooth Activations
Arindam Banerjee, Pedro Cisneros-Velarde, Libin Zhu +1
We consider the problem of optimization of deep learning models with smooth activation functions. While there exist influential results on the problem from the ``near initializatio…
A Universal Trade-off Between the Model Size, Test Loss, and Training Loss of Linear Predictors
Nikhil Ghosh, Mikhail Belkin
In this work we establish an algorithm and distribution independent non-asymptotic trade-off between the model size, excess test loss, and training loss of linear predictors. Speci…
Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting
Neil Mallinar, James B. Simon, Amirhesam Abedsoltan +3
The practical success of overparameterized neural networks has motivated the recent scientific study of interpolating methods, which perfectly fit their training data. Certain inte…
A note on Linear Bottleneck networks and their Transition to Multilinearity
Libin Zhu, Parthe Pandit, Mikhail Belkin
Randomly initialized wide neural networks transition to linear functions of weights as the width grows, in a ball of radius around initialization. A necessary condition for…
On the Inconsistency of Kernel Ridgeless Regression in Fixed Dimensions
Daniel Beaglehole, Mikhail Belkin, Parthe Pandit
``Benign overfitting'', the ability of certain algorithms to interpolate noisy training data and yet perform well out-of-sample, has been a topic of considerable recent interest. W…