Towards an Understanding of Residual Networks Using Neural Tangent Hierarchy (NTH)

Yuqing Li, Tao Luo, Nung Kwan Yip

Gradient descent yields zero training loss in polynomial time for deep neural networks despite non-convex nature of the objective function. The behavior of network in the infinite…