Regularization and Reparameterization Avoid Vanishing Gradients in Sigmoid-Type Networks

Leni Ven, Johannes Lederer

Deep learning requires several design choices, such as the nodes' activation functions and the widths, types, and arrangements of the layers. One consideration when making these ch…