53 citations · 54 across the 3 of their papers we have counts for
3 papers
cs.DS2023
A Whole New Ball Game: A Primal Accelerated Method for Matrix Games and Minimizing the Maximum of Smooth Functions
Yair Carmon, Arun Jambulapati, Yujia Jin +1
We design algorithms for minimizing over a -dimensional Euclidean or simplex domain. When each is -Lipschitz and -smooth, our method computes…
cs.LG2023★ 1 cited
Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond
Itai Kreisler, Mor Shpigel Nacson, Daniel Soudry +1
Recent research shows that when Gradient Descent (GD) is applied to neural networks, the loss almost never decreases monotonically. Instead, the loss oscillates as gradient descent…
math.OC2016★ 53 cited
Accelerated Methods for Non-Convex Optimization
Yair Carmon, John C. Duchi, Oliver Hinder +1
We present an accelerated gradient method for non-convex optimization problems with Lipschitz continuous first and second derivatives. The method requires time $O(ε^{-7/4} \log(1/…