signADAM: Learning Confidences for Deep Neural Networks

Dong Wang, Yicheng Liu, Wenwo Tang +4

In this paper, we propose a new first-order gradient-based algorithm to train deep neural networks. We first introduce the sign operation of stochastic gradients (as in sign-based…