SANE: The phases of gradient descent through Sharpness Adjusted Number of Effective parameters

Lawrence Wang, Stephen J. Roberts

Modern neural networks are undeniably successful. Numerous studies have investigated how the curvature of loss landscapes can affect the quality of solutions. In this work we consi…