Invariance of Weight Distributions in Rectified MLPs

Russell Tsuchida, Farbod Roosta-Khorasani, Marcus Gallagher

An interesting approach to analyzing neural networks that has received renewed attention is to examine the equivalent kernel of the neural network. This is based on the fact that a…