Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes

James Lucas, Juhan Bae, Michael R. Zhang +3

Linear interpolation between initial neural network parameters and converged parameters after training with stochastic gradient descent (SGD) typically leads to a monotonic decreas…