6 papers · 1 filter
Optimal Asynchronous Stochastic Nonconvex Optimization under Heavy-Tailed Noise
Yidong Wu, Luo Luo
This paper considers the problem of asynchronous stochastic nonconvex optimization with heavy-tailed gradient noise and arbitrarily heterogeneous computation times across workers.…
Near-Optimal Decentralized Stochastic Nonconvex Optimization with Heavy-Tailed Noise
Menglian Wang, Zhuanghua Liu, Luo Luo
This paper studies decentralized stochastic nonconvex optimization problem over row-stochastic networks. We consider the heavy-tailed gradient noise which is empirically observed i…
Explicit Global Convergence Rates of BFGS without Line Search
Jianjiang Yu, Weiguo Gao, Luo Luo
This paper studies the convergence rates of the Broyden--Fletcher--Goldfarb--Shanno~(BFGS) method without line search. We show that the BFGS method with an adaptive step size [Gao…
A Parameter-Free and Near-Optimal Zeroth-Order Algorithm for Stochastic Convex Optimization
Kunjie Ren, Luo Luo
This paper considers zeroth-order optimization for stochastic convex minimization problem. We propose a parameter-free stochastic zeroth-order method (POEM) by introducing a step-s…
Decentralized Gradient-Free Methods for Stochastic Non-Smooth Non-Convex Optimization
Zhenwei Lin, Jingfan Xia, Qi Deng +1
We consider decentralized gradient-free optimization of minimizing Lipschitz continuous functions that satisfy neither smoothness nor convexity assumption. We propose two novel gra…
Optimizing over Multiple Distributions under Generalized Quasar-Convexity Condition
Shihong Ding, Long Yang, Luo Luo +1
We study a typical optimization model where the optimization variable is composed of multiple probability distributions. Though the model appears frequently in practice, such as fo…