Deep neural networks with dependent weights: Gaussian Process mixture limit, heavy tails, sparsity and compressibility

Hoil Lee, Fadhel Ayed, Paul Jung +3

This article studies the infinite-width limit of deep feedforward neural networks whose weights are dependent, and modelled via a mixture of Gaussian distributions. Each hidden nod…