Improved weight initialization for deep and narrow feedforward neural network

Hyunwoo Lee, Yunho Kim, Seung Yeop Yang +1

Appropriate weight initialization settings, along with the ReLU activation function, have become cornerstones of modern deep learning, enabling the training and deployment of highl…