3 papers
cs.LG2026
Stochastic Gradient Descent with Momentum is Algorithmically Stable
Yunwen Lei, Zimeng Wang, Xiaoming Yuan
Stochastic gradient descent with momentum (SGDM) is one of the most widely used optimization algorithms in machine learning. While optimization properties of SGDM have been extensi…
cs.LG2026
Learning Theory of the SVRG: Generalization and Convergence Analysis
Yunwen Lei, Zimeng Wang, Xiaoming Yuan
Variance reduction (VR) methods employ stochastic gradients with decreasing variance, and they have been widely applied to solve large-scale optimization problems in machine learni…
math.OC2024
A Single-Loop Stochastic Proximal Quasi-Newton Method for Large-Scale Nonsmooth Convex Optimization
Yongcun Song, Zimeng Wang, Xiaoming Yuan +1
We propose a new stochastic proximal quasi-Newton method for minimizing the sum of two convex functions in the particular context that one of the functions is the average of a larg…