Stepping on the Edge: Curvature Aware Learning Rate Tuners

Vincent Roulet, Atish Agarwala, Jean-Bastien Grill +3

Curvature information -- particularly, the largest eigenvalue of the loss Hessian, known as the sharpness -- often forms the basis for learning rate tuners. However, recent work ha…