Adam${}^{+}$: A Stochastic Method with Adaptive Variance Reduction

Mingrui Liu, Wei Zhang, Francesco Orabona +1

Adam is a widely used stochastic optimization method for deep learning applications. While practitioners prefer Adam because it requires less parameter tuning, its use is problemat…