Stochastic Quasi-Gradient Methods: Variance Reduction via Jacobian Sketching

Robert M. Gower, Peter Richtárik, Francis Bach

We develop a new family of variance reduced stochastic gradient descent methods for minimizing the average of a very large number of smooth functions. Our method --JacSketch-- is m…