Normalized Direction-preserving Adam

Zijun Zhang, Lin Ma, Zongpeng Li +1

Adaptive optimization algorithms, such as Adam and RMSprop, have shown better optimization performance than stochastic gradient descent (SGD) in some scenarios. However, recent stu…