2 citations · 6 across the 10 of their papers we have counts for
11 papers
Random Reshuffling for Smooth Convex Optimization: Dominates Stochastic Gradient Descent
Zijian Liu
Stochastic Gradient Descent (SGD) is one of the most classical optimization algorithms with favorable theoretical guarantees, yet its practical implementation differs subtly from i…
Random Reshuffling Dominates Stochastic Gradient Descent
Zijian Liu
Stochastic Gradient Descent () is one of the most classical optimization algorithms with favorable theoretical guarantees, yet the practical implementation of $\texts…
Improved Last-Iterate Convergence of Shuffling Gradient Methods for Nonsmooth Convex Optimization
Zijian Liu, Zhengyuan Zhou
We study the convergence of the shuffling gradient method, a popular algorithm employed to minimize the finite-sum function with regularization, in which functions are passed to ap…
Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping
Zijian Liu, Zhengyuan Zhou
Recently, the study of heavy-tailed noises in first-order nonconvex stochastic optimization has gotten a lot of attention since it was recognized as a more realistic condition as s…
On the Last-Iterate Convergence of Shuffling Gradient Methods
Zijian Liu, Zhengyuan Zhou
Shuffling gradient methods are widely used in modern machine learning tasks and include three popular implementations: Random Reshuffle (RR), Shuffle Once (SO), and Incremental Gra…
Revisiting the Last-Iterate Convergence of Stochastic Gradient Methods
Zijian Liu, Zhengyuan Zhou
In the past several years, the last-iterate convergence of the Stochastic Gradient Descent (SGD) algorithm has triggered people's interest due to its good performance in practice b…
Stochastic Nonsmooth Convex Optimization with Heavy-Tailed Noises: High-Probability Bound, In-Expectation Rate and Initial Distance Adaptation
Zijian Liu, Zhengyuan Zhou
Recently, several studies consider the stochastic optimization problem but in a heavy-tailed noise regime, i.e., the difference between the stochastic gradient and the true gradien…
High Probability Convergence of Stochastic Gradient Methods
Zijian Liu, Ta Duy Nguyen, Thien Hang Nguyen +2
In this work, we describe a generic approach to show convergence with high probability for both stochastic convex and non-convex optimization with sub-Gaussian noise. In previous w…
Breaking the Lower Bound with (Little) Structure: Acceleration in Non-Convex Stochastic Optimization with Heavy-Tailed Noise
Zijian Liu, Jiawei Zhang, Zhengyuan Zhou
We consider the stochastic optimization problem with smooth but not necessarily convex objectives in the heavy-tailed noise regime, where the stochastic gradient's noise is assumed…
Near-Optimal Non-Convex Stochastic Optimization under Generalized Smoothness
Zijian Liu, Srikanth Jagabathula, Zhengyuan Zhou
The generalized smooth condition, -smoothness, has triggered people's interest since it is more realistic in many optimization problems shown by both empirical and t…
META-STORM: Generalized Fully-Adaptive Variance Reduced SGD for Unbounded Functions
Zijian Liu, Ta Duy Nguyen, Thien Hang Nguyen +2
We study the application of variance reduction (VR) techniques to general non-convex stochastic optimization problems. In this setting, the recent work STORM [Cutkosky-Orabona '19]…