Guaranteed Sufficient Decrease for Stochastic Variance Reduced Gradient Optimization

Fanhua Shang, Yuanyuan Liu, Kaiwen Zhou +3

In this paper, we propose a novel sufficient decrease technique for stochastic variance reduced gradient descent methods such as SVRG and SAGA. In order to make sufficient decrease…