Subset Selection with Shrinkage: Sparse Linear Modeling when the SNR is low

Rahul Mazumder, Peter Radchenko, Antoine Dedieu

We study a seemingly unexpected and relatively less understood overfitting aspect of a fundamental tool in sparse linear modeling - best subset selection, which minimizes the resid…