Improving Deep Learning Optimization through Constrained Parameter Regularization

Jörg K. H. Franke, Michael Hefenbrock, Gregor Koehler +1

Regularization is a critical component in deep learning. The most commonly used approach, weight decay, applies a constant penalty coefficient uniformly across all parameters. This…