High-Probability Convergence of SGD via Batched Updates

Feng Zhu, Robert W. Heath, Aritra Mitra

Stochastic gradient descent (SGD) is the primary workhorse for large-scale optimization. While the average behavior of its iterates, typically characterized by mean-squared error b…