Momentum Centering and Asynchronous Update for Adaptive Gradient Methods

Juntang Zhuang, Yifan Ding, Tommy Tang +3

We propose ACProp (Asynchronous-centering-Prop), an adaptive optimizer which combines centering of second momentum and asynchronous update (e.g. for $t$-th update, denominator uses…