On uniform-in-time diffusion approximation for stochastic gradient descent

Lei Li, Yuliang Wang

The diffusion approximation of stochastic gradient descent (SGD) in current literature is only valid on a finite time interval. In this paper, we establish the uniform-in-time diff…