Estimating Implicit Regularization in Deep Learning

Joseph H. Rudoler, Kevin Tan, Giles Hooker +1

Deep learning systems are known to exhibit implicit regularization (alt. implicit bias), favoring simple solutions instead of merely minimizing the loss function. In some cases, we…