Deep Information Propagation

Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli +1

We study the behavior of untrained neural networks whose weights and biases are randomly distributed using mean field theory. We show the existence of depth scales that naturally l…