Momentum Residual Neural Networks

Michael E. Sander, Pierre Ablin, Mathieu Blondel +1

The training of deep residual neural networks (ResNets) with backpropagation has a memory cost that increases linearly with respect to the depth of the network. A way to circumvent…