38 citations · 66 across the 4 of their papers we have counts for
4 papers
Disentangling the Mechanisms Behind Implicit Regularization in SGD
Zachary Novack, Simran Kaur, Tanya Marwah +2
A number of competing hypotheses have been proposed to explain why small-batch Stochastic Gradient Descent (SGD)leads to improved generalization over the full-batch regime, with re…
On the Maximum Hessian Eigenvalue and Generalization
Simran Kaur, Jeremy Cohen, Zachary C. Lipton
The mechanisms by which certain training interventions, such as increasing learning rates and applying batch normalization, improve the generalization of deep networks remains a my…
Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability
Jeremy M. Cohen, Simran Kaur, Yuanzhi Li +2
We empirically demonstrate that full-batch gradient descent on neural network training objectives typically operates in a regime we call the Edge of Stability. In this regime, the…
Are Perceptually-Aligned Gradients a General Property of Robust Classifiers?
Simran Kaur, Jeremy Cohen, Zachary C. Lipton
For a standard convolutional neural network, optimizing over the input pixels to maximize the score of some target class will generally produce a grainy-looking version of the orig…