Stochastic gradient with least-squares control variates

Fabio Nobile, Matteo Raviola, Nathan Schaeffer

The stochastic gradient descent (SGD) method is a widely used approach for solving stochastic optimization problems, but its convergence is typically slow. Existing variance reduct…