Neural Networks as Kernel Learners: The Silent Alignment Effect

Alexander Atanasov, Blake Bordelon, Cengiz Pehlevan

Neural networks in the lazy training regime converge to kernel machines. Can neural networks in the rich feature learning regime learn a kernel machine with a data-dependent kernel…