Predicting the outputs of finite deep neural networks trained with noisy gradients

Gadi Naveh, Oded Ben-David, Haim Sompolinsky +1

A recent line of works studied wide deep neural networks (DNNs) by approximating them as Gaussian Processes (GPs). A DNN trained with gradient flow was shown to map to a GP governe…