Regularization by Misclassification in ReLU Neural Networks

Elisabetta Cornacchia, Jan Hązła, Ido Nachum +1

We study the implicit bias of ReLU neural networks trained by a variant of SGD where at each step, the label is changed with probability $p$ to a random label (label smoothing bein…