Activated Gradients for Deep Neural Networks

Mei Liu, Liangming Chen, Xiaohao Du +2

Deep neural networks often suffer from poor performance or even training failure due to the ill-conditioned problem, the vanishing/exploding gradient problem, and the saddle point…