Neural ODEs as the Deep Limit of ResNets with constant weights

Benny Avelin, Kaj Nyström

In this paper we prove that, in the deep limit, the stochastic gradient descent on a ResNet type deep neural network, where each layer shares the same weight matrix, converges to t…