Implicit Sparse Regularization: The Impact of Depth and Early Stopping

Jiangyuan Li, Thanh V. Nguyen, Chinmay Hegde +1

In this paper, we study the implicit bias of gradient descent for sparse regression. We extend results on regression with quadratic parametrization, which amounts to depth-2 diagon…