Valid post-selection inference
arXiv:1306.1059 · doi:10.1214/12-AOS1077
Abstract
It is common practice in statistical data analysis to perform data-driven variable selection and derive statistical inference from the resulting model. Such inference enjoys none of the guarantees that classical statistical theory provides for tests and confidence intervals when the model has been chosen a priori. We propose to produce valid ``post-selection inference'' by reducing the problem to one of simultaneous inference and hence suitably widening conventional confidence and retention intervals. Simultaneity is required for all linear functions that arise as coefficient estimates in all submodels. By purchasing ``simultaneity insurance'' for all possible submodels, the resulting post-selection inference is rendered universally valid under all possible model selection procedures. This inference is therefore generally conservative for particular selection procedures, but it is always less conservative than full Scheffe protection. Importantly it does not depend on the truth of the selected submodel, and hence it produces valid inference even in wrong models. We describe the structure of the simultaneous inference problem and give some asymptotic results.
arXiv commentPublished in at http://dx.doi.org/10.1214/12-AOS1077 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)
References in corpus (9)
- Matching Methods for Causal Inference: A Review and a Look Forward
- Valid post-selection inference
- Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator
- Can one estimate the conditional distribution of post-model-selection estimators?
- Confidence Sets Based on Penalized Maximum Likelihood Estimators in Gaussian Regression
- The distribution of model averaging estimators and an impossibility result regarding its estimation
- The distribution of a linear predictor after model selection: Unconditional finite-sample distributions and asymptotic approximations
- On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding
- Distributional Results for Thresholding Estimators in High-Dimensional Gaussian Regression Models
Cited by in corpus (127)
- On asymptotically optimal confidence regions and tests for high-dimensional models
- Valid post-selection inference
- Controlling the false discovery rate via knockoffs
- Exact post-selection inference, with application to the lasso
- Model Selection Techniques -- An Overview
- Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations
- SLOPE - Adaptive variable selection via convex optimization
- Optimal Inference After Model Selection
- High-Dimensional Inference: Confidence Intervals, -Values and R-Software hdi
- Valid Post-Selection and Post-Regularization Inference: An Elementary, General Approach
- Adaptive Concentration of Regression Trees, with Application to Random Forests
- Confidence intervals for high-dimensional inverse covariance estimation
- Regularized Ordinal Regression and the ordinalNet R Package
- Exact Post Model Selection Inference for Marginal Screening
- Rates of convergence of the Adaptive LASSO estimators to the Oracle distribution and higher order refinements by the bootstrap
- Least Squares Approximation for a Distributed System
- Exact Post-Selection Inference for Sequential Regression Procedures
- Post hoc inference via joint family-wise error rate control
- Simultaneous Control of All False Discovery Proportions in Large-Scale Multiple Hypothesis Testing
- On Various Confidence Intervals Post-Model-Selection
- Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection
- Simultaneous high-probability bounds on the false discovery proportion in structured, regression, and online settings
- Discovering and Deciphering Relationships Across Disparate Data Modalities
- Only Closed Testing Procedures are Admissible for Controlling False Discovery Proportions
- Selective inference for effect modification via the lasso
- Inference in High Dimensions with the Penalized Score Test
- A Simple Adjustment for Bandwidth Snooping
- Selective inference with a randomized response
- Bootstrapping and Sample Splitting For High-Dimensional, Assumption-Free Inference
- Confounder selection strategies targeting stable treatment effect estimators
- A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression
- Rejoinder: "A significance test for the lasso"
- More Powerful Conditional Selective Inference for Generalized Lasso by Parametric Programming
- Discussion: "A significance test for the lasso"
- On the Post Selection Inference constant under Restricted Isometry Properties
- Two Sample Inference for Populations of Graphical Models with Applications to Functional Connectivity
- Evaluating the Effectiveness of Personalized Medicine with Software
- A Theory of Statistical Inference for Ensuring the Robustness of Scientific Results
- Nonparametric Empirical Bayes Estimation on Heterogeneous Data
- A Bootstrap Lasso + Partial Ridge Method to Construct Confidence Intervals for Parameters in High-dimensional Sparse Linear Models
- Spherical Cap Packing Asymptotics and Rank-Extreme Detection
- MAGIC: a general, powerful and tractable method for selective inference
- Post-selection estimation and testing following aggregated association tests
- Uniformly Valid Confidence Sets Based on the Lasso
- Floodgate: inference for model-free variable importance
- Fast and Interpretable Consensus Clustering via Minipatch Learning
- AdaPT: An interactive procedure for multiple testing with side information
- Discussion: "A significance test for the lasso"
- Controlling False Discovery Rate Using Gaussian Mirrors
- The costs and benefits of uniformly valid causal inference with high-dimensional nuisance parameters
- Confidence Sets Based on Thresholding Estimators in High-Dimensional Gaussian Regression Models
- Orthogonal Machine Learning: Power and Limitations
- An efficient and robust approach to Mendelian randomization with measured pleiotropic effects in a high-dimensional setting
- Graphical tools for selecting conditional instrumental sets
- Discussion: "A significance test for the lasso"
- Valid Post-selection Inference in Assumption-lean Linear Regression
- Low-Complexity Methods for Estimation After Parameter Selection
- Exact Post-Selection Inference for Changepoint Detection and Other Generalized Lasso Problems
- Confidence intervals centered on bootstrap smoothed estimators
- Models as Approximations II: A Model-Free Theory of Parametric Regression
- Likelihood Ratio Test in Multivariate Linear Regression: from Low to High Dimension
- Tractable Post-Selection Maximum Likelihood Inference for the Lasso
- Proximal MCMC for Bayesian Inference of Constrained and Regularized Estimation
- On Selecting and Conditioning in Multiple Testing and Selective Inference
- Forward-Selected Panel Data Approach for Program Evaluation
- Using Regression Kernels to Forecast A Failure to Appear in Court
- On Hodges' Superefficiency and Merits of Oracle Property in Model Selection
- Optimal multiple testing and design in clinical trials
- Asymptotics of selective inference
- Valid Post-Detection Inference for Change Points Identified Using Trend Filtering
- Informative Goodness-of-Fit for Multivariate Distributions
- EigenPrism: Inference for High-Dimensional Signal-to-Noise Ratios
- A Groupwise Approach for Inferring Heterogeneous Treatment Effects in Causal Inference
- Constructing confidence sets after lasso selection by randomized estimator augmentation
- Non-Bayesian Post-Model-Selection Estimation as Estimation Under Model Misspecification
- Discovery and inference of a causal network with hidden confounding
- Two-Stage Robust and Sparse Distributed Statistical Inference for Large-Scale Data
- More Powerful and General Selective Inference for Stepwise Feature Selection using the Homotopy Continuation Approach
- Post hoc false positive control for spatially structured hypotheses
- Information, Privacy and Stability in Adaptive Data Analysis
- Beyond subjective and objective in statistics
- Covariate Selection Based on a Assumpton-free Approach to Linear Regression with Exact Probabilities
- A Lasso-OLS Hybrid Approach to Covariate Selection and Average Treatment Effect Estimation for Clustered RCTs Using Design-Based Methods
- Post-selection inference with HSIC-Lasso
- Fitting High-Dimensional Interaction Models with Error Control
- Use Of Vapnik-Chervonenkis Dimension in Model Selection
- Uniformly Valid Inference Based on the Lasso in Linear Mixed Models
- Inference after black box selection
- Post hoc false discovery proportion inference under a Hidden Markov Model
- Admissibility of the usual confidence set for the mean of a univariate or bivariate normal population: The unknown-variance case
- A Normality Test for High-dimensional Data based on a Nearest Neighbor Approach
- Post-selection inference for linear mixed model parameters using the conditional Akaike information criterion
- Algorithm-agnostic significance testing in supervised learning with multimodal data
- Model selection for estimation of causal parameters
- Valid uncertainty quantification about the model in a linear regression setting
- Unlucky Number 13? Manipulating Evidence Subject to Snooping
- A Multi-resolution Theory for Approximating Infinite--Zero-: Transitional Inference, Individualized Predictions, and a World Without Bias-Variance Trade-off
- Improving Power by Conditioning on Less in Post-selection Inference for Changepoints
- Markov Neighborhood Regression for High-Dimensional Inference
- Selection-adjusted inference: an application to confidence intervals for cis-eQTL effect sizes
- Post model-fitting exploration via a "Next-Door" analysis
- Prediction out-of-sample using block shrinkage estimators: model selection and predictive inference
- Submodularity in Statistics: Comparing the Success of Model Selection Methods
- Selective Inference in Propensity Score Analysis
- Mathematical modelling, selection and hierarchical inference to determine the minimal dose in IFN therapy against Myeloproliferative Neoplasms
- Selective inference after convex clustering with penalization
- Post-Model-Selection Statistical Inference with Interrupted Time Series Designs: An Evaluation of an Assault Weapons Ban in California
- Inference for High Dimensional Censored Quantile Regression
- Statistical Inference in High-dimensional Generalized Linear Models with Streaming Data
- Weak Signal Identification and Inference in Penalized Model Selection
- Excess Optimism: How Biased is the Apparent Error of an Estimator Tuned by SURE?
- Selective Inference and Learning Mixed Graphical Models
- revisit: a Workflow Tool for Data Science
- On overfitting and post-selection uncertainty assessments
- An Efficient Post-Selection Inference on High-Order Interaction Models
- Selective Inference for Latent Block Models
- DebiNet: Debiasing Linear Models with Nonlinear Overparameterized Neural Networks
- Inject Machine Learning into Significance Test for Misspecified Linear Models
- All of Linear Regression
- Spatial Variable Selection and An Application to Virginia Lyme Disease Emergence
- Confidently Comparing Estimators with the c-value
- Selective Inference Approach for Statistically Sound Predictive Pattern Mining
- Selective inference after likelihood- or test-based model selection in linear models
- A Computationally Efficient Approach to False Discovery Rate Control and Power Maximisation via Randomisation and Mirror Statistic
- Kernel Ordinary Differential Equations
- Bayesian Selective Inference: Non-informative Priors
- Post-selection Inference in Regression Models for Group Testing Data