4 papers · 1 filter
Optimal Two-Step Stepsize Schedule for Stochastic Gradient Methods
Luwei Bai, Baoyu Zhou
Structured nonconstant large stepsizes can improve the convergence of gradient descent in the deterministic setting. However, in stochastic optimization, aggressive stepsizes can a…
Generalization of Silver Stepsize Schedule to Stochastic Optimization
Luwei Bai, Yang Zeng, Baoyu Zhou
This work introduces a two-step stepsize schedule for stochastic gradient methods minimizing smooth strongly convex functions. We consider the setting where only stochastic gradien…
Avoiding strict saddle points of nonconvex regularized problems
Luwei Bai, Yaohua Hu, Hao Wang +1
In this paper, we consider a class of non-convex and non-smooth sparse optimization problems, which encompass most existing nonconvex sparsity-inducing terms. We show the second-or…
Anderson Acceleration in Nonsmooth Problems: Local Convergence via Active Manifold Identification
Kexin Li, Luwei Bai, Xiao Wang +1
Anderson acceleration is an effective technique for enhancing the efficiency of fixed-point iterations; however, analyzing its convergence in nonsmooth settings presents significan…