Convergence and Implicit Regularization Properties of Gradient Descent for Deep Residual Networks

Rama Cont, Alain Rossier, RenYuan Xu

We prove linear convergence of gradient descent to a global optimum for the training of deep residual networks with constant layer width and smooth activation function. We show tha…