Normalization effects on deep neural networks

Jiahui Yu, Konstantinos Spiliopoulos

We study the effect of normalization on the layers of deep neural networks of feed-forward type. A given layer $i$ with $N_i$ hidden units is allowed to be normalized by $1/N_{i}…