Normalization effects on shallow neural networks and related asymptotic expansions

Jiahui Yu, Konstantinos Spiliopoulos

We consider shallow (single hidden layer) neural networks and characterize their performance when trained with stochastic gradient descent as the number of hidden units $N$ and gra…