Bayesian Inference with Deep Weakly Nonlinear Networks

Boris Hanin, Alexander Zlokapa

We show at a physics level of rigor that Bayesian inference with a fully connected neural network and a shaped nonlinearity of the form $\varphi (t)=t+\psi t^3/L$ is (perturbatively) solv…