Stochastic Gradient Descent Revisited

Azar Louzi

Stochastic gradient descent (SGD) has been a go-to algorithm for nonconvex stochastic optimization problems arising in machine learning. Its theory however often requires a strong…