Linear regression with overparameterized linear neural networks: Tight upper and lower bounds for implicit ${\ell }^1$-regularization

Hannes Matt, Dominik Stöger

Modern machine learning models are often trained in a setting where the number of parameters exceeds the number of training samples. To understand the implicit bias of gradient des…