13 citations · 18 across the 20 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023
On the Interplay Between Stepsize Tuning and Progressive Sharpening
Vincent Roulet, Atish Agarwala, Fabian Pedregosa
Recent empirical work has revealed an intriguing property of deep learning models by which the sharpness (largest eigenvalue of the Hessian) increases throughout optimization until…
cs.LG2023
SAM operates far from home: eigenvalue regularization as a dynamical phenomenon
Atish Agarwala, Yann N. Dauphin
The Sharpness Aware Minimization (SAM) optimization algorithm has been shown to control large eigenvalues of the loss Hessian and provide generalization benefits in a variety of se…