Tracking the gradients using the Hessian: A new look at variance reducing stochastic methods

Robert M. Gower, Nicolas Le Roux, Francis Bach

Our goal is to improve variance reducing stochastic methods through better control variates. We first propose a modification of SVRG which uses the Hessian to track gradients over…