168 citations · 795 across the 28 of their papers we have counts for
4 papers · 1 filter
NeurIPS 2020 Competition: Predicting Generalization in Deep Learning
Yiding Jiang, Pierre Foret, Scott Yak +7
Understanding generalization in deep learning is arguably one of the most important questions in deep learning. Deep learning has been successfully adopted to a large number of pro…
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy
Edward Moroshko, Suriya Gunasekar, Blake Woodworth +3
We provide a detailed asymptotic study of gradient flow trajectories and their implicit optimization bias when minimizing the exponential loss over "diagonal linear networks". This…
Mirrorless Mirror Descent: A Natural Derivation of Mirror Descent
Suriya Gunasekar, Blake Woodworth, Nathan Srebro
We present a primal only derivation of Mirror Descent as a "partial" discretization of gradient flow on a Riemannian manifold where the metric tensor is the Hessian of the Mirror D…
Kernel and Rich Regimes in Overparametrized Models
Blake Woodworth, Suriya Gunasekar, Jason D. Lee +5
A recent line of work studies overparametrized neural networks in the "kernel regime," i.e. when the network behaves during training as a kernelized linear predictor, and thus trai…