Deep Control - a simple automatic gain control for memory efficient and high performance training of deep convolutional neural networks

Brendan Ruff

Training a deep convolutional neural net typically starts with a random initialisation of all filters in all layers which severely reduces the forward signal and back-propagated er…