10 citations · 23 across the 7 of their papers we have counts for
3 papers · 1 filter
Convex and Non-convex Optimization Under Generalized Smoothness
Haochuan Li, Jian Qian, Yi Tian +2
Classical analysis of convex and non-convex optimization methods often requires the Lipshitzness of the gradient, which limits the analysis to functions bounded by quadratics. Rece…
Convergence of Adam Under Relaxed Assumptions
Haochuan Li, Alexander Rakhlin, Ali Jadbabaie
In this paper, we provide a rigorous proof of convergence of the Adaptive Moment Estimate (Adam) algorithm for a wide class of optimization objectives. Despite the popularity and e…
Variance-reduced Clipping for Non-convex Optimization
Amirhossein Reisizadeh, Haochuan Li, Subhro Das +1
Gradient clipping is a standard training technique used in deep learning applications such as large-scale language modeling to mitigate exploding gradients. Recent experimental stu…