Sparsity-depth Tradeoff in Infinitely Wide Deep Neural Networks

Chanwoo Chun, Daniel D. Lee

We investigate how sparse neural activity affects the generalization performance of a deep Bayesian neural network at the large width limit. To this end, we derive a neural network…