5 papers · 1 filter
Optimal Recursive Composition and Dyadic Phase Laws for Gradient Descent with Predetermined Stepsizes
Yu Liu, Kang Chen, Rujun Jiang +1
Predetermined stepsize schedules featuring carefully chosen long steps have recently been shown to accelerate gradient descent (GD) on smooth convex functions. A prominent class of…
The Exact Time-Uniform Rate Frontier for Stochastic Gradient Descent on Smooth Convex Objectives
Ruijie Li, Kang Chen, Tianyu Wang
We study the time-uniform convergence of the raw iterate of standard stochastic gradient descent (SGD) for unconstrained smooth convex objectives. We prove that, under standard noi…
Revisiting Stochastic Gradient Descent for Strongly Convex Objectives: Tight Uniform-in-Time Bounds
Kang Chen, Yasong Feng, Tianyu Wang
Stochastic optimization via Stochastic Gradient Descent (SGD) is a fundamental problem in statistics and optimization. This paper revisits Stochastic Gradient Descent (SGD) for str…
Breaking a Logarithmic Barrier in the Stopping Time Convergence Rate of Stochastic First-order Methods
Yasong Feng, Yifan Jiang, Tianyu Wang +1
This work provides a novel convergence analysis for stochastic optimization in terms of stopping times, addressing the practical reality that algorithms are often terminated adapti…
The Anytime Convergence of Stochastic Gradient Descent with Momentum: From a Continuous-Time Perspective
Yasong Feng, Yifan Jiang, Tianyu Wang +1
We study the stochastic optimization problem from a continuous-time perspective, with a focus on the Stochastic Gradient Descent with Momentum (SGDM) method. We show that the traje…