5 citations · 6 across the 10 of their papers we have counts for
10 papers
Dual Acceleration for Minimax Optimization: Linear Convergence Under Relaxed Assumptions
Jingwang Li, Xiao Li
This paper addresses the bilinearly coupled minimax optimization problem: $\min_{x \in \mathbb{R}^{d_x}}\max_{y \in \mathbb{R}^{d_y}} \ f_1(x) + f_2(x) + y^{\top} Bx - g_1(y) - g_2…
A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD
Ruinan Jin, Xiao Li, Yaoliang Yu +1
Adaptive Moment Estimation (Adam) is a cornerstone optimization algorithm in deep learning, widely recognized for its flexibility with adaptive learning rates and efficiency in han…
A KL-based Analysis Framework with Applications to Non-Descent Optimization Methods
Junwen Qiu, Bohao Ma, Xiao Li +1
We propose a novel analysis framework for non-descent-type optimization methodologies in nonconvex scenarios based on the Kurdyka-Lojasiewicz property. Our framework allows coverin…
A Generalized Version of Chung's Lemma and its Applications
Li Jiang, Xiao Li, Andre Milzarek +1
Chung's Lemma is a classical tool for establishing asymptotic convergence rates of (stochastic) optimization methods under strong convexity-type assumptions and appropriate polynom…
A New Random Reshuffling Method for Nonsmooth Nonconvex Finite-sum Optimization
Junwen Qiu, Xiao Li, Andre Milzarek
Random reshuffling techniques are prevalent in large-scale applications, such as training neural networks. While the convergence and acceleration effects of random reshuffling-type…
High Probability Guarantees for Random Reshuffling
Hengxu Yu, Xiao Li
We consider the stochastic gradient method with random reshuffling () for tackling smooth nonconvex optimization problems. finds broad applications in pr…
Distributed Stochastic Optimization under a General Variance Condition
Kun Huang, Xiao Li, Shi Pu
Distributed stochastic optimization has drawn great attention recently due to its effectiveness in solving large-scale machine learning problems. Though numerous algorithms have be…
A Unified Convergence Theorem for Stochastic Optimization Methods
Xiao Li, Andre Milzarek
In this work, we provide a fundamental unified convergence theorem used for deriving expected and almost sure convergence results for a series of stochastic optimization methods. O…
Distributed Random Reshuffling over Networks
Kun Huang, Xiao Li, Andre Milzarek +2
In this paper, we consider distributed optimization problems where agents, each possessing a local cost function, collaboratively minimize the average of the local cost functio…
Convergence of Random Reshuffling Under The Kurdyka-Łojasiewicz Inequality
Xiao Li, Andre Milzarek, Junwen Qiu
We study the random reshuffling (RR) method for smooth nonconvex optimization problems with a finite-sum structure. Though this method is widely utilized in practice such as the tr…