Learning Sparse Networks Using Targeted Dropout

Aidan N. Gomez, Ivan Zhang, Siddhartha Rao Kamalakara +4

Neural networks are easier to optimise when they have many more weights than are required for modelling the mapping from inputs to outputs. This suggests a two-stage learning proce…