Vanishing Nodes: Another Phenomenon That Makes Training Deep Neural Networks Difficult

Wen-Yu Chang, Tsung-Nan Lin

It is well known that the problem of vanishing/exploding gradients is a challenge when training deep networks. In this paper, we describe another phenomenon, called vanishing nodes…