Revisiting the Neural Tangent Kernel: the role of large width and depth

William St-Arnaud, Margarida Carvalho, Golnoosh Farnadi

Overparameterized fully-connected neural networks have been shown to behave like kernel models when trained with gradient descent, assuming standard scaling conditions on the width…