The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization

Mufan Bill Li, Mihai Nica, Daniel M. Roy

Theoretical results show that neural networks can be approximated by Gaussian processes in the infinite-width limit. However, for fully connected networks, it has been previously s…