Functional Large Deviations for Wide Deep Neural Networks with Gaussian Initialization and Lipschitz Activations

Claudio Macci, Barbara Pacchiarotti, Katerina Papagiannouli +2

We establish a functional large deviation principle for fully connected multi-layer perceptrons with i.i.d. Gaussian weights (LeCun initialization) and general Lipschitz activation…