Accelerated Stochastic Gradient Descent for Minimizing Finite Sums
arXiv:1506.03016
Abstract
We propose an optimization method for minimizing the finite sums of smooth convex functions. Our method incorporates an accelerated gradient descent (AGD) and a stochastic variance reduction gradient (SVRG) in a mini-batch setting. Unlike SVRG, our method can be directly applied to non-strongly and strongly convex problems. We show that our method achieves a lower overall complexity than the recently proposed methods that supports non-strongly convex problems. Moreover, this method has a fast rate of convergence for strongly convex problems. Our experiments show the effectiveness of our method.
[v2] corrected citation to proxSVRG, corrected typos in Figure 1(option2) and 3(R4 -> R3)