Scale Normalization

Henry Z. Lo, Kevin Amaral, Wei Ding

One of the difficulties of training deep neural networks is caused by improper scaling between layers. Scaling issues introduce exploding / gradient problems, and have typically be…