Implicit Regularization for Group Sparsity

Jiangyuan Li, Thanh V. Nguyen, Chinmay Hegde +1

We study the implicit regularization of gradient descent towards structured sparsity via a novel neural reparameterization, which we call a diagonally grouped linear neural network…