4 papers
Stochastic Saddle Avoidance Beyond Unit Excitation and Smoothness: A Pathwise Lyapunov-Perron Framework
Junwen Qiu, Bohao Ma, Andre Milzarek +1
Unit excitation (UE) is a common assumption in stochastic saddle avoidance: the stochastic error must have a uniformly positive component along every direction, in expectation. Thi…
A KL-based Analysis Framework with Applications to Non-Descent Optimization Methods
Junwen Qiu, Bohao Ma, Xiao Li +1
We propose a novel analysis framework for non-descent-type optimization methodologies in nonconvex scenarios based on the Kurdyka-Lojasiewicz property. Our framework allows coverin…
A Generalized Version of Chung's Lemma and its Applications
Li Jiang, Xiao Li, Andre Milzarek +1
Chung's Lemma is a classical tool for establishing asymptotic convergence rates of (stochastic) optimization methods under strong convexity-type assumptions and appropriate polynom…
Convergence of SGD with momentum in the nonconvex case: A time window-based analysis
Junwen Qiu, Bohao Ma, Andre Milzarek
The stochastic gradient descent method with momentum (SGDM) is a common approach for solving large-scale and stochastic optimization problems. Despite its popularity, the convergen…