Neural network optimization strategies and the topography of the loss landscape

Jianneng Yu, Alexandre V. Morozov

Neural networks are trained by optimizing multi-dimensional sets of fitting parameters on non-convex loss landscapes. Low-loss regions of the landscapes correspond to the parameter…