Lasso, knockoff and Gaussian covariates: a comparison

Abstract

Given data $\mathbf{y}$ and $k$ covariates ${\mathbf{x}}_j$ one problem in linear regression is to decide which if any of the covariates to include when regressing the dependent variable $\mathbf{y}$ on the covariates ${\mathbf{x}}_j$. In this paper three such methods, lasso, knockoff and Gaussian covariates are compared using simulations and real data. The Gaussian covariate method is based on exact probabilities which are valid for all $\mathbf{y}$ and ${\mathbf{x}}_j$ making it model free. Moreover the probabilities agree with those based on the F-distribution for the standard linear model with i.i.d. Gaussian errors. It is conceptually, mathematically and algorithmically very simple, it is very fast and makes no use of simulations. It outperforms lasso and knockoff in all respects by a considerable margin.

The output of runcomp.R has been deleted. New versions of selvar.f and selvar.R are provided which are faster than the old ones. The paper is now 32 pages long