Tensor Programs II: Neural Tangent Kernel for Any Architecture

Greg Yang

We prove that a randomly initialized neural network of *any architecture* has its Tangent Kernel (NTK) converge to a deterministic limit, as the network widths tend to infinity. We…