Why bigger is not always better: on finite and infinite neural networks

Laurence Aitchison

Recent work has argued that neural networks can be understood theoretically by taking the number of channels to infinity, at which point the outputs become Gaussian process (GP) di…