On the diffusion approximation of nonconvex stochastic gradient descent

Wenqing Hu, Chris Junchi Li, Lei Li +1

We study the Stochastic Gradient Descent (SGD) method in nonconvex optimization problems from the point of view of approximating diffusion processes. We prove rigorously that the d…