Gradual DropIn of Layers to Train Very Deep Neural Networks

Leslie N. Smith, Emily M. Hand, Timothy Doster

We introduce the concept of dynamically growing a neural network during training. In particular, an untrainable deep network starts as a trainable shallow network and newly added l…