2 citations · 6 across the 10 of their papers we have counts for
6 papers · 1 filter
Random Reshuffling for Smooth Convex Optimization: Dominates Stochastic Gradient Descent
Zijian Liu
Stochastic Gradient Descent (SGD) is one of the most classical optimization algorithms with favorable theoretical guarantees, yet its practical implementation differs subtly from i…
Random Reshuffling Dominates Stochastic Gradient Descent
Zijian Liu
Stochastic Gradient Descent () is one of the most classical optimization algorithms with favorable theoretical guarantees, yet the practical implementation of $\texts…
Improved Last-Iterate Convergence of Shuffling Gradient Methods for Nonsmooth Convex Optimization
Zijian Liu, Zhengyuan Zhou
We study the convergence of the shuffling gradient method, a popular algorithm employed to minimize the finite-sum function with regularization, in which functions are passed to ap…
Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping
Zijian Liu, Zhengyuan Zhou
Recently, the study of heavy-tailed noises in first-order nonconvex stochastic optimization has gotten a lot of attention since it was recognized as a more realistic condition as s…
Stochastic Nonsmooth Convex Optimization with Heavy-Tailed Noises: High-Probability Bound, In-Expectation Rate and Initial Distance Adaptation
Zijian Liu, Zhengyuan Zhou
Recently, several studies consider the stochastic optimization problem but in a heavy-tailed noise regime, i.e., the difference between the stochastic gradient and the true gradien…
High Probability Convergence of Stochastic Gradient Methods
Zijian Liu, Ta Duy Nguyen, Thien Hang Nguyen +2
In this work, we describe a generic approach to show convergence with high probability for both stochastic convex and non-convex optimization with sub-Gaussian noise. In previous w…