A significance test for the lasso

Abstract

In the sparse linear regression setting, we consider testing the significance of the predictor variable that enters the current lasso model, in the sequence of models visited along the lasso solution path. We propose a simple test statistic based on lasso fitted values, called the covariance test statistic, and show that when the true model is linear, this statistic has an $\mathrm{Exp}(1)$ asymptotic distribution under the null hypothesis (the null being that all truly active variables are contained in the current lasso model). Our proof of this result for the special case of the first predictor to enter the model (i.e., testing for a single significant predictor variable against the global null) requires only weak assumptions on the predictor matrix $X$. On the other hand, our proof for a general step in the lasso path places further technical assumptions on $X$ and the generative model, but still allows for the important high-dimensional case $p>n$, and does not necessarily require that the current lasso model achieves perfect recovery of the truly active variables. Of course, for testing the significance of an additional variable between two nested linear models, one typically uses the chi-squared test, comparing the drop in residual sum of squares (RSS) to a ${\chi }_1^2$ distribution. But when this additional variable is not fixed, and has been chosen adaptively or greedily, this test is no longer appropriate: adaptivity makes the drop in RSS stochastically much larger than ${\chi }_1^2$ under the null hypothesis. Our analysis explicitly accounts for adaptivity, as it must, since the lasso builds an adaptive sequence of linear models as the tuning parameter $\lambda$ decreases. In this analysis, shrinkage plays a key role: though additional variables are chosen adaptively, the coefficients of lasso active variables are shrunken due to the ${\ell }_1$ penalty. Therefore, the test statistic (which is based on lasso fitted values) is in a sense balanced by these two opposing properties - adaptivity and shrinkage - and its null distribution is tractable and asymptotically $\mathrm{Exp}(1)$.

Published in at http://dx.doi.org/10.1214/13-AOS1175 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (11)

• Pathwise coordinate optimization
• On the "degrees of freedom" of the lasso
• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• High-dimensional variable selection
• Near-ideal model selection by ${\ell }_1$ minimization
• Degrees of freedom in lasso problems
• Statistical significance in high-dimensional linear models
• Validity of the expected Euler characteristic heuristic
• P-values for high-dimensional regression
• Adaptive testing for the graphical lasso
• Sequential Selection Procedures and False Discovery Rate Control

Cited by in corpus (72)

• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• Controlling the false discovery rate via knockoffs
• Exact post-selection inference, with application to the lasso
• SLOPE - Adaptive variable selection via convex optimization
• Optimal Inference After Model Selection
• High-Dimensional Inference: Confidence Intervals, $p$-Values and R-Software hdi
• Valid Post-Selection and Post-Regularization Inference: An Elementary, General Approach
• Granger Causality in Multi-variate Time Series using a Time Ordered Restricted Vector Autoregressive Model
• Inference and Uncertainty Quantification for Noisy Matrix Completion
• Electricity Price Forecasting using Sale and Purchase Curves: The X-Model
• Regularized Ordinal Regression and the ordinalNet R Package
• High-dimensional inference in misspecified linear models
• Exact Post-Selection Inference for Sequential Regression Procedures
• Network classification with applications to brain connectomics
• Goodness of fit tests for high-dimensional linear models
• Statistical Inference, Learning and Models in Big Data
• A significance test for forward stepwise model selection
• Prediction error of cross-validated Lasso
• Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
• How much does your data exploration overfit? Controlling bias via information usage
• Weak Signals in the Mobility Landscape: Car Sharing in Ten European Cities
• On efficient adjustment in causal graphs
• Inference in High Dimensions with the Penalized Score Test
• Estimating the error variance in a high-dimensional linear model
• Statistical Inferences for Polarity Identification in Natural Language
• A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression
• Discussion: "A significance test for the lasso"
• EMD-regression for modelling multi-scale relationships, and application to weather-related cardiovascular mortality
• Evaluating the Effectiveness of Personalized Medicine with Software
• Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation
• A Bootstrap Lasso + Partial Ridge Method to Construct Confidence Intervals for Parameters in High-dimensional Sparse Linear Models
• Adaptive testing for the graphical lasso
• Sequential Selection Procedures and False Discovery Rate Control
• Scalable Sparse Cox's Regression for Large-Scale Survival Data via Broken Adaptive Ridge
• Inference for a Large Directed Acyclic Graph with Unspecified Interventions
• Iterative Hard Thresholding for Model Selection in Genome-Wide Association Studies
• False Discovery Rate Control Under General Dependence By Symmetrized Data Aggregation
• Estimating the Lasso's Effective Noise
• Using Regression Kernels to Forecast A Failure to Appear in Court
• Quantum Algorithms for the Pathwise Lasso
• Feature-specific inference for penalized regression using local false discovery rates
• Tractable Post-Selection Maximum Likelihood Inference for the Lasso
• Group-bound: confidence intervals for groups of variables in sparse high-dimensional regression without assumptions on the design
• Valid Post-Detection Inference for Change Points Identified Using Trend Filtering
• More Powerful and General Selective Inference for Stepwise Feature Selection using the Homotopy Continuation Approach
• High-dimensional genome-wide association study and misspecified mixed model analysis
• Two-Stage Robust and Sparse Distributed Statistical Inference for Large-Scale Data
• Confidence Region of Singular Subspaces for Low-rank Matrix Regression
• SuRF: a New Method for Sparse Variable Selection, with Application in Microbiome Data Analysis
• Variable Selection with Rigorous Uncertainty Quantification using Deep Bayesian Neural Networks: Posterior Concentration and Bernstein-von Mises Phenomenon
• The Benefit of Group Sparsity in Group Inference with De-biased Scaled Group Lasso
• Sample Splitting and Weak Assumption Inference For Time Series
• A Lasso-OLS Hybrid Approach to Covariate Selection and Average Treatment Effect Estimation for Clustered RCTs Using Design-Based Methods
• Marginal false discovery rates for penalized regression models
• Blockwise and coordinatewise thresholding to combine tests of different natures in modern ANOVA
• Simultaneous prediction and community detection for networks with application to neuroimaging
• Uncertainty Quantification Under Group Sparsity
• Lasso, knockoff and Gaussian covariates: a comparison
• Demystifying the Bias from Selective Inference: a Revisit to Dawid's Treatment Selection Problem
• The Terminating-Random Experiments Selector: Fast High-Dimensional Variable Selection with False Discovery Rate Control
• Sparsified Simultaneous Confidence Intervals for High-Dimensional Linear Models
• Statistical significance in high-dimensional linear mixed models
• Higher Criticism Tuned Regression For Weak And Sparse Signals
• Selective Confidence Intervals for Martingale Regression Model
• Confidently Comparing Estimators with the c-value
• Logistic regression and Ising networks: prediction and estimation when violating lasso assumptions
• Selective Inference via Marginal Screening for High Dimensional Classification
• Instrument variable detection with graph learning : an application to high dimensional GIS-census data for house pricing
• Fast Markov chain Monte Carlo for high dimensional Bayesian regression models with shrinkage priors
• Wavelet Screaming: a novel approach to analyzing GWAS data
• Ultra High Dimensional Change Point Detection
• High-Dimensional Inference Based on the Leave-One-Covariate-Out LASSO Path