55 citations · 484 across the 40 of their papers we have counts for
9 papers · 1 filter
Toward Understanding the Impact of Staleness in Distributed Machine Learning
Wei Dai, Yi Zhou, Nanqing Dong +2
Many distributed machine learning (ML) systems adopt the non-synchronous execution in order to alleviate the network communication bottleneck, resulting in stale parameters that do…
SpiderBoost and Momentum: Faster Stochastic Variance Reduction Algorithms
Zhe Wang, Kaiyi Ji, Yi Zhou +2
SARAH and SPIDER are two recently developed stochastic variance-reduced algorithms, and SPIDER has been shown to achieve a near-optimal first-order oracle complexity in smooth nonc…
Cubic Regularization with Momentum for Nonconvex Optimization
Zhe Wang, Yi Zhou, Yingbin Liang +1
Momentum is a popular technique to accelerate the convergence in practical training, and its impact on convergence guarantee has been well-studied for first-order algorithms. Howev…
Asynchronous decentralized accelerated stochastic gradient descent
Guanghui Lan, Yi Zhou
In this work, we introduce an asynchronous decentralized accelerated stochastic gradient descent type of method for decentralized stochastic optimization, considering communication…
A Note on Inexact Condition for Cubic Regularized Newton's Method
Zhe Wang, Yi Zhou, Yingbin Liang +1
This note considers the inexact cubic-regularized Newton's method (CR), which has been shown in \cite{Cartis2011a} to achieve the same order-level convergence rate to a secondary s…
Convergence of Cubic Regularization for Nonconvex Optimization under KL Property
Yi Zhou, Zhe Wang, Yingbin Liang
Cubic-regularized Newton's method (CR) is a popular algorithm that guarantees to produce a second-order stationary solution for solving nonconvex optimization problems. However, ex…