Towards Better Generalization: BP-SVRG in Training Deep Neural Networks

Hao Jin, Dachao Lin, Zhihua Zhang

Stochastic variance-reduced gradient (SVRG) is a classical optimization method. Although it is theoretically proved to have better convergence performance than stochastic gradient…