Wide Neural Networks with Bottlenecks are Deep Gaussian Processes

Devanshu Agrawal, Theodore Papamarkou, Jacob Hinkle

There has recently been much work on the "wide limit" of neural networks, where Bayesian neural networks (BNNs) are shown to converge to a Gaussian process (GP) as all hidden layer…