On the Algorithmic Stability and Generalization of Adaptive Optimization Methods

Han Nguyen, Hai Pham, Sashank J. Reddi +1

Despite their popularity in deep learning and machine learning in general, the theoretical properties of adaptive optimizers such as Adagrad, RMSProp, Adam or AdamW are not yet ful…