Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses

Charles G. Frye, James Simon, Neha S. Wadia +3

Despite the fact that the loss functions of deep neural networks are highly non-convex, gradient-based optimization algorithms converge to approximately the same performance from m…