AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks

Garrett Bingham, Risto Miikkulainen

Neural networks require careful weight initialization to prevent signals from exploding or vanishing. Existing initialization schemes solve this problem in specific cases by assumi…