An Optimistic Acceleration of AMSGrad for Nonconvex Optimization

Jun-Kun Wang, Xiaoyun Li, Belhal Karimi +1

We propose a new variant of AMSGrad, a popular adaptive gradient based optimization algorithm widely used for training deep neural networks. Our algorithm adds prior knowledge abou…