Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis

Hyunwoo Lee, Hayoung Choi, Hyunju Kim

As a neural network's depth increases, it can improve generalization performance. However, training deep networks is challenging due to gradient and signal propagation issues. To a…