8 papers
Sign-Based Optimizers Are Effective Under Heavy-Tailed Noise
Dingzhi Yu, Hongyi Tao, Yuanyu Wan +2
While adaptive gradient methods are the workhorse of modern machine learning, sign-based optimization algorithms such as Lion and Muon have recently demonstrated superior empirical…
Optimal Asynchronous Stochastic Nonconvex Optimization under Heavy-Tailed Noise
Yidong Wu, Luo Luo
This paper considers the problem of asynchronous stochastic nonconvex optimization with heavy-tailed gradient noise and arbitrarily heterogeneous computation times across workers.…
Near-Optimal Decentralized Stochastic Nonconvex Optimization with Heavy-Tailed Noise
Menglian Wang, Zhuanghua Liu, Luo Luo
This paper studies decentralized stochastic nonconvex optimization problem over row-stochastic networks. We consider the heavy-tailed gradient noise which is empirically observed i…
Accelerated Evolving Set Processes for Local PageRank Computation
Binbin Huang, Luo Luo, Yanghua Xiao +2
This work proposes a novel framework based on nested evolving set processes to accelerate Personalized PageRank (PPR) computation. At each stage of the process, we employ a localiz…
Explicit Global Convergence Rates of BFGS without Line Search
Jianjiang Yu, Weiguo Gao, Luo Luo
This paper studies the convergence rates of the Broyden--Fletcher--Goldfarb--Shanno~(BFGS) method without line search. We show that the BFGS method with an adaptive step size [Gao…
A Parameter-Free and Near-Optimal Zeroth-Order Algorithm for Stochastic Convex Optimization
Kunjie Ren, Luo Luo
This paper considers zeroth-order optimization for stochastic convex minimization problem. We propose a parameter-free stochastic zeroth-order method (POEM) by introducing a step-s…