58 citations · 219 across the 25 of their papers we have counts for
6 papers · 1 filter
ASVRG: Accelerated Proximal SVRG
Fanhua Shang, Licheng Jiao, Kaiwen Zhou +3
This paper proposes an accelerated proximal stochastic variance reduced gradient (ASVRG) method, in which we design a simple and effective momentum acceleration trick. Unlike most…
Norm-Range Partition: A Universal Catalyst for LSH based Maximum Inner Product Search (MIPS)
Xiao Yan, Xinyan Dai, Jie Liu +2
Recently, locality sensitive hashing (LSH) was shown to be effective for MIPS and several algorithms including -ALSH, Sign-ALSH and Simple-LSH have been proposed. In this pape…
A Simple Stochastic Variance Reduced Algorithm with Fast Convergence Rates
Kaiwen Zhou, Fanhua Shang, James Cheng
Recent years have witnessed exciting progress in the study of stochastic variance reduced gradient methods (e.g., SVRG, SAGA), their accelerated variants (e.g, Katyusha) and their…
Direct Acceleration of SAGA using Sampled Negative Momentum
Kaiwen Zhou
Variance reduction is a simple and effective technique that accelerates convex (or non-convex) stochastic optimization. Among existing variance reduction methods, SVRG and SAGA ado…
Guaranteed Sufficient Decrease for Stochastic Variance Reduced Gradient Optimization
Fanhua Shang, Yuanyuan Liu, Kaiwen Zhou +3
In this paper, we propose a novel sufficient decrease technique for stochastic variance reduced gradient descent methods such as SVRG and SAGA. In order to make sufficient decrease…
VR-SGD: A Simple Stochastic Variance Reduction Method for Machine Learning
Fanhua Shang, Kaiwen Zhou, Hongying Liu +5
In this paper, we propose a simple variant of the original SVRG, called variance reduced stochastic gradient descent (VR-SGD). Unlike the choices of snapshot and starting points in…