A Sober Look at Neural Network Initializations

Ingo Steinwart

Initializing the weights and the biases is a key part of the training process of a neural network. Unlike the subsequent optimization phase, however, the initialization phase has g…