Strong Gaussian Approximation of Constant Learning Rate SGD: Bias and Trajectory Inference

Joseph Keslin, Wei Biao Wu

This paper develops strong Gaussian approximations (KMT type) for constant learning rate stochastic gradient descent (SGD). We work with heavy tailed noise and assume that the numb…