5 citations · 8 across the 10 of their papers we have counts for
10 papers
Stochastic Saddle Avoidance Beyond Unit Excitation and Smoothness: A Pathwise Lyapunov-Perron Framework
Junwen Qiu, Bohao Ma, Andre Milzarek +1
Unit excitation (UE) is a common assumption in stochastic saddle avoidance: the stochastic error must have a uniformly positive component along every direction, in expectation. Thi…
A KL-based Analysis Framework with Applications to Non-Descent Optimization Methods
Junwen Qiu, Bohao Ma, Xiao Li +1
We propose a novel analysis framework for non-descent-type optimization methodologies in nonconvex scenarios based on the Kurdyka-Lojasiewicz property. Our framework allows coverin…
A Generalized Version of Chung's Lemma and its Applications
Li Jiang, Xiao Li, Andre Milzarek +1
Chung's Lemma is a classical tool for establishing asymptotic convergence rates of (stochastic) optimization methods under strong convexity-type assumptions and appropriate polynom…
Convergence of SGD with momentum in the nonconvex case: A time window-based analysis
Junwen Qiu, Bohao Ma, Andre Milzarek
The stochastic gradient descent method with momentum (SGDM) is a common approach for solving large-scale and stochastic optimization problems. Despite its popularity, the convergen…
Random Reshuffling with Momentum: Complexity Bounds and Last-iterate Convergence
Junwen Qiu, Bohao Ma, Andre Milzarek
Random reshuffling with momentum (RRM) corresponds to the SGD optimizer with the 'momentum' option enabled, as found in many machine learning libraries such as PyTorch and TensorFl…
A New Random Reshuffling Method for Nonsmooth Nonconvex Finite-sum Optimization
Junwen Qiu, Xiao Li, Andre Milzarek
Random reshuffling techniques are prevalent in large-scale applications, such as training neural networks. While the convergence and acceleration effects of random reshuffling-type…
A Normal Map-Based Proximal Stochastic Gradient Method: Convergence and Identification Properties
Junwen Qiu, Li Jiang, Andre Milzarek
The proximal stochastic gradient method (PSGD) is one of the state-of-the-art approaches for stochastic composite-type problems. In contrast to its deterministic counterpart, PSGD…
A Unified Convergence Theorem for Stochastic Optimization Methods
Xiao Li, Andre Milzarek
In this work, we provide a fundamental unified convergence theorem used for deriving expected and almost sure convergence results for a series of stochastic optimization methods. O…
Distributed Random Reshuffling over Networks
Kun Huang, Xiao Li, Andre Milzarek +2
In this paper, we consider distributed optimization problems where agents, each possessing a local cost function, collaboratively minimize the average of the local cost functio…
Convergence of Random Reshuffling Under The Kurdyka-Łojasiewicz Inequality
Xiao Li, Andre Milzarek, Junwen Qiu
We study the random reshuffling (RR) method for smooth nonconvex optimization problems with a finite-sum structure. Though this method is widely utilized in practice such as the tr…