Mean Field Theory of Activation Functions in Deep Neural Networks

Mirco Milletarí, Thiparat Chotibut, Paolo E. Trevisanutto

We present a Statistical Mechanics (SM) model of deep neural networks, connecting the energy-based and the feed forward networks (FFN) approach. We infer that FFN can be understood…