10 citations · 23 across the 7 of their papers we have counts for
7 papers
Convex and Non-convex Optimization Under Generalized Smoothness
Haochuan Li, Jian Qian, Yi Tian +2
Classical analysis of convex and non-convex optimization methods often requires the Lipshitzness of the gradient, which limits the analysis to functions bounded by quadratics. Rece…
Convergence of Adam Under Relaxed Assumptions
Haochuan Li, Alexander Rakhlin, Ali Jadbabaie
In this paper, we provide a rigorous proof of convergence of the Adaptive Moment Estimate (Adam) algorithm for a wide class of optimization objectives. Despite the popularity and e…
Variance-reduced Clipping for Non-convex Optimization
Amirhossein Reisizadeh, Haochuan Li, Subhro Das +1
Gradient clipping is a standard training technique used in deep learning applications such as large-scale language modeling to mitigate exploding gradients. Recent experimental stu…
On Convergence of Gradient Descent Ascent: A Tight Local Analysis
Haochuan Li, Farzan Farnia, Subhro Das +1
Gradient Descent Ascent (GDA) methods are the mainstream algorithms for minimax optimization in generative adversarial networks (GANs). Convergence properties of GDA have drawn sig…
Byzantine-Robust Federated Linear Bandits
Ali Jadbabaie, Haochuan Li, Jian Qian +1
In this paper, we study a linear bandit optimization problem in a federated setting where a large collection of distributed agents collaboratively learn a common linear bandit mode…
Neural Network Weights Do Not Converge to Stationary Points: An Invariant Measure Perspective
Jingzhao Zhang, Haochuan Li, Suvrit Sra +1
This work examines the deep disconnect between existing theoretical analyses of gradient-based algorithms and the practice of training deep neural networks. Specifically, we provid…
Complexity Lower Bounds for Nonconvex-Strongly-Concave Min-Max Optimization
Haochuan Li, Yi Tian, Jingzhao Zhang +1
We provide a first-order oracle complexity lower bound for finding stationary points of min-max optimization problems where the objective function is smooth, nonconvex in the minim…