Type-II Saddles and Probabilistic Stability of Stochastic Gradient Descent

Liu Ziyin, Botao Li, Tomer Galanti +1

Characterizing and understanding the dynamics of stochastic gradient descent (SGD) around saddle points remains an open problem. We first show that saddle points in neural networks…