4 papers
Adaptive Newton-CG methods with global and local analysis for unconstrained optimization with Hölder continuous Hessian
Ziyang Zeng, Junyu Zhang, Chuan He
In this paper, we study Newton-conjugate gradient (Newton-CG) methods for minimizing a nonconvex function whose Hessian is -Hölder continuous with modulus and…
Bias-Variance Trade-off for Clipped Stochastic First-Order Methods: From Bounded Variance to Infinite Mean
Chuan He
Stochastic optimization is fundamental to modern machine learning. Recent research has extended the study of stochastic first-order methods (SFOMs) from light-tailed to heavy-taile…
Accelerated stochastic first-order method for convex optimization under heavy-tailed noise
Chuan He, Zhaosong Lu
We study convex composite optimization problems, where the objective function is given by the sum of a prox-friendly function and a convex function whose subgradients are estimated…
Complexity of normalized stochastic first-order methods with momentum under heavy-tailed noise
Chuan He, Zhaosong Lu, Defeng Sun +1
In this paper, we propose practical normalized stochastic first-order methods with Polyak momentum, multi-extrapolated momentum, and recursive momentum for solving unconstrained op…