2 citations · 3 across the 5 of their papers we have counts for
5 papers
General framework for online-to-nonconvex conversion: Schedule-free SGD is also effective for nonconvex optimization
Kwangjun Ahn, Gagik Magakyan, Ashok Cutkosky
This work investigates the effectiveness of schedule-free methods, developed by A. Defazio et al. (NeurIPS 2024), in nonconvex optimization settings, inspired by their remarkable e…
Reevaluating Theoretical Analysis Methods for Optimization in Deep Learning
Hoang Tran, Qinzi Zhang, Ashok Cutkosky
There is a significant gap between our theoretical understanding of optimization algorithms used in deep learning and their practical performance. Theoretical development usually f…
Private Zeroth-Order Nonsmooth Nonconvex Optimization
Qinzi Zhang, Hoang Tran, Ashok Cutkosky
We introduce a new zeroth-order algorithm for private stochastic optimization on nonconvex and nonsmooth objectives. Given a dataset of size , our algorithm ensures …
Adam with model exponential moving average is effective for nonconvex optimization
Kwangjun Ahn, Ashok Cutkosky
In this work, we offer a theoretical analysis of two modern optimization techniques for training large and complex models: (i) adaptive optimization algorithms, such as Adam, and (…
Random Scaling and Momentum for Non-smooth Non-convex Optimization
Qinzi Zhang, Ashok Cutkosky
Training neural networks requires optimizing a loss function that may be highly irregular, and in particular neither convex nor smooth. Popular training algorithms are based on sto…