How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD

Pierfrancesco Beneventano, Andrea Pinto, Tomaso Poggio

We investigate the ability of deep neural networks to identify the support of the target function. Our findings reveal that mini-batch SGD effectively learns the support in the fir…