On weight initialization in deep neural networks

Siddharth Krishna Kumar

A proper initialization of the weights in a neural network is critical to its convergence. Current insights into weight initialization come primarily from linear activation functio…