SVRG and Beyond via Posterior Correction

Nico Daheim, Thomas Möllenhoff, Ming Liang Ang +1

Stochastic Variance Reduced Gradient (SVRG) and its variants aim to speed-up training by using gradient corrections. Originally proposed over a decade ago, these methods have never…