On the Stability of the Jacobian Matrix in Deep Neural Networks

Benjamin Dadoun, Soufiane Hayou, Hanan Salam +2

Deep neural networks are known to suffer from exploding or vanishing gradients as depth increases, a phenomenon closely tied to the spectral behavior of the input-output Jacobian.…