Information-theoretic reduction of deep neural networks to linear models in the overparametrized proportional regime

Francesco Camilli, Daria Tieplova, Eleonora Bergamin +1

We rigorously analyse fully-trained neural networks of arbitrary depth in the Bayesian optimal setting in the so-called proportional scaling regime where the number of training sam…