Demystifying the Myths and Legends of Nonconvex Convergence of SGD

Aritra Dutta, El Houcine Bergou, Soumia Boucherouite +3

Stochastic gradient descent (SGD) and its variants are the main workhorses for solving large-scale optimization problems with nonconvex objective functions. Although the convergenc…