2 papers
cs.LG2024
General framework for online-to-nonconvex conversion: Schedule-free SGD is also effective for nonconvex optimization
Kwangjun Ahn, Gagik Magakyan, Ashok Cutkosky
This work investigates the effectiveness of schedule-free methods, developed by A. Defazio et al. (NeurIPS 2024), in nonconvex optimization settings, inspired by their remarkable e…
cs.LG2024
Adam with model exponential moving average is effective for nonconvex optimization
Kwangjun Ahn, Ashok Cutkosky
In this work, we offer a theoretical analysis of two modern optimization techniques for training large and complex models: (i) adaptive optimization algorithms, such as Adam, and (…