Variance Reduction Methods Do Not Need to Compute Full Gradients: Improved Efficiency through Shuffling

Daniil Medyakov, Gleb Molodtsov, Savelii Chezhegov +2

Stochastic optimization algorithms are widely used for machine learning with large-scale data. However, their convergence often suffers from non-vanishing variance. Variance Reduct…