Convergence of Markov Chains for Constant Step-size Stochastic Gradient Descent with Separable Functions

David Shirokoff, Philip Zaleski

Stochastic gradient descent (SGD) is a popular algorithm for minimizing objective functions that arise in machine learning. For constant step-sized SGD, the iterates form a Markov…