Confidence Intervals and Hypothesis Testing for High-Dimensional Regression

Abstract

Fitting high-dimensional statistical models often requires the use of non-linear parameter estimation procedures. As a consequence, it is generally impossible to obtain an exact characterization of the probability distribution of the parameter estimates. This in turn implies that it is extremely challenging to quantify the \emph{uncertainty} associated with a certain parameter estimate. Concretely, no commonly accepted procedure exists for computing classical measures of uncertainty and statistical significance as confidence intervals or $p$-values for these models. We consider here high-dimensional linear regression problem, and propose an efficient algorithm for constructing confidence intervals and $p$-values. The resulting confidence intervals have nearly optimal size. When testing for the null hypothesis that a certain parameter is vanishing, our method has nearly optimal power. Our approach is based on constructing a `de-biased' version of regularized M-estimators. The new construction improves over recent work in the field in that it does not assume a special structure on the design matrix. We test our method on synthetic data and a high-throughput genomic data set about riboflavin production rate.

40 pages, 4 pdf figures

References in corpus (3)

• Nearly unbiased variable selection under minimax concave penalty
• Asymptotic normality and optimalities in estimation of large Gaussian graphical models
• Residual variance and the signal-to-noise ratio in high-dimensional linear models

Cited by in corpus (171)

• Controlling the false discovery rate via knockoffs
• Exact post-selection inference, with application to the lasso
• SLOPE - Adaptive variable selection via convex optimization
• High-Dimensional Inference: Confidence Intervals, $p$-Values and R-Software hdi
• Inference and Uncertainty Quantification for Noisy Matrix Completion
• High-dimensional regression adjustments in randomized experiments
• Exact Post Model Selection Inference for Marginal Screening
• Global and Simultaneous Hypothesis Testing for High-Dimensional Logistic Regression Models
• Distributed Estimation and Inference with Statistical Guarantees
• A review of distributed statistical inference
• High-dimensional inference in misspecified linear models
• Exact Post-Selection Inference for Sequential Regression Procedures
• Double/Debiased Machine Learning for Treatment and Causal Parameters
• Goodness of fit tests for high-dimensional linear models
• Communication-efficient sparse regression: a one-shot approach
• Statistical Inference, Learning and Models in Big Data
• The Lasso with general Gaussian designs with applications to hypothesis testing
• Efficient Distributed Learning with Sparsity
• High Dimensional Expectation-Maximization Algorithm: Statistical Optimization and Asymptotic Normality
• Distributed High-dimensional Regression Under a Quantile Loss Function
• Prediction error of cross-validated Lasso
• Statistical Inference for Model Parameters in Stochastic Gradient Descent
• How much does your data exploration overfit? Controlling bias via information usage
• Significance testing in non-sparse high-dimensional linear models
• Selective inference for effect modification via the lasso
• The Asymptotic Distribution of the MLE in High-dimensional Logistic Models: Arbitrary Covariance
• Inferring Graphs from Cascades: A Sparse Recovery Framework
• Estimating the error variance in a high-dimensional linear model
• Geometric Inference for General High-Dimensional Linear Inverse Problems
• Statistical unfolding of elementary particle spectra: Empirical Bayes estimation and bias-corrected uncertainty quantification
• Recursive Sparse Point Process Regression with Application to Spectrotemporal Receptive Field Plasticity Analysis
• Testability of high-dimensional linear models with non-sparse structures
• Uncertainty Quantification for Matrix Compressed Sensing and Quantum Tomography Problems
• Linearly-involved Moreau-Enhanced-over-Subspace Model: Debiased Sparse Modeling and Stable Outlier-Robust Regression
• Regularized Estimation and Testing for High-Dimensional Multi-Block Vector-Autoregressive Models
• Model-assisted inference for treatment effects using regularized calibrated estimation with high-dimensional data
• A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression
• On Semiparametric Exponential Family Graphical Models
• Rejoinder: "A significance test for the lasso"
• De-Biasing The Lasso With Degrees-of-Freedom Adjustment
• High-dimensional empirical likelihood inference
• Two Sample Inference for Populations of Graphical Models with Applications to Functional Connectivity
• Penalized Maximum Likelihood Estimation of Multi-layered Gaussian Graphical Models
• Regression Analysis for Microbiome Compositional Data
• Post-Regularization Inference for Time-Varying Nonparanormal Graphical Models
• Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation
• Uniform Inference for High-dimensional Quantile Regression: Linear Functionals and Regression Rank Scores
• Provable Algorithms for Inference in Topic Models
• De-biased sparse PCA: Inference and testing for eigenstructure of large covariance matrices
• Communication-efficient Distributed Sparse Linear Discriminant Analysis
• A Bootstrap Lasso + Partial Ridge Method to Construct Confidence Intervals for Parameters in High-dimensional Sparse Linear Models
• High-Dimensional Sparse Linear Bandits
• Confidence Intervals for Causal Effects with Invalid Instruments using Two-Stage Hard Thresholding with Voting
• Asymptotic Confidence Regions for High-dimensional Structured Sparsity
• Floodgate: inference for model-free variable importance
• Uniformly Valid Confidence Sets Based on the Lasso
• Statistical Inferences of Linear Forms for Noisy Matrix Completion
• Uniformly Valid Post-Regularization Confidence Regions for Many Functional Parameters in Z-Estimation Framework
• Bayesian Sparse Linear Regression with Unknown Symmetric Error
• Debiased distributed learning for sparse partial linear models in high dimensions
• Integrative High Dimensional Multiple Testing with Heterogeneity under Data Sharing Constraints
• Second order Stein: SURE for SURE and other applications in high-dimensional inference
• Inference for Sparse Conditional Precision Matrices
• Inference for Individual Mediation Effects and Interventional Effects in Sparse High-Dimensional Causal Graphical Models
• Constrained High Dimensional Statistical Inference
• The costs and benefits of uniformly valid causal inference with high-dimensional nuisance parameters
• Non-Asymptotic Guarantees for Reliable Identification of Granger Causality via the LASSO
• Inference for the Case Probability in High-dimensional Logistic Regression
• False Discovery Rate Control via Data Splitting
• Debiased Inference of Average Partial Effects in Single-Index Models
• Estimation and Inference for High Dimensional Generalized Linear Models: A Splitting and Smoothing Approach
• A Revisit to De-biased Lasso for Generalized Linear Models
• Inference for high-dimensional linear mixed-effects models: A quasi-likelihood approach
• Breaking the curse of dimensionality in regression
• Inference In High-dimensional Single-Index Models Under Symmetric Designs
• Prediction regions through Inverse Regression
• Method of Contraction-Expansion (MOCE) for Simultaneous Inference in Linear Models
• High Dimensional M-Estimation with Missing Outcomes: A Semi-Parametric Framework
• Individual Data Protected Integrative Regression Analysis of High-dimensional Heterogeneous Data
• Efficient Distributed Estimation of Inverse Covariance Matrices
• Inference for Heteroskedastic PCA with Missing Data
• Confidence Intervals for High-Dimensional Linear Regression: Minimax Rates and Adaptivity
• High Dimensional Linear GMM
• Inter-Subject Analysis: Inferring Sparse Interactions with Dense Intra-Graphs
• Discussion: "A significance test for the lasso"
• Joint Estimation and Inference for Data Integration Problems based on Multiple Multi-layered Gaussian Graphical Models
• Distributed Bootstrap for Simultaneous Inference Under High Dimensionality
• Sparsity Double Robust Inference of Average Treatment Effects
• Generalized Linear Models with Linear Constraints for Microbiome Compositional Data
• Distributed Multitask Learning
• Bandit Algorithms for Precision Medicine
• Two-sample inference for high-dimensional Markov networks
• Semi-Supervised Empirical Risk Minimization: Using unlabeled data to improve prediction
• Accurate Inference for Adaptive Linear Models
• Linear regression and its inference on noisy network-linked data
• Estimating the Lasso's Effective Noise
• Surrogate Assisted Semi-supervised Inference for High Dimensional Risk Prediction
• Adaptive Inferential Method for Monotone Graph Invariants
• Bias-Aware Inference in Regularized Regression Models
• Feature-specific inference for penalized regression using local false discovery rates
• High-Dimensional Confidence Regions in Sparse MRI
• Proximal MCMC for Bayesian Inference of Constrained and Regularized Estimation
• Hierarchical correction of p-values via an ultrametric tree running Ornstein-Uhlenbeck process
• Local Projection Inference in High Dimensions
• Two-Stage Robust and Sparse Distributed Statistical Inference for Large-Scale Data
• On the efficiency of the de-biased Lasso
• Communication-efficient Distributed Estimation and Inference for Transelliptical Graphical Models
• Statistical control for spatio-temporal MEG/EEG source imaging with desparsified multi-task Lasso
• Approximate Newton-based statistical inference using only stochastic gradients
• High dimensional regression and matrix estimation without tuning parameters
• EigenPrism: Inference for High-Dimensional Signal-to-Noise Ratios
• ${\ell }_1$-penalized Multinomial Regression: Estimation, inference, and prediction, with an application to risk factor identification for different dementia subtypes
• Detangling robustness in high dimensions: composite versus model-averaged estimation
• Lasso under Multi-way Clustering: Estimation and Post-selection Inference
• Many Average Partial Effects: with An Application to Text Regression
• Design-based theory for Lasso adjustment in randomized block experiments and rerandomized experiments
• Uncertainty Quantification for High Dimensional Sparse Nonparametric Additive Models
• Nonparametric Inference via Bootstrapping the Debiased Estimator
• Information Directed Sampling for Sparse Linear Bandits
• Randomized tests for high-dimensional regression: A more efficient and powerful solution
• The Benefit of Group Sparsity in Group Inference with De-biased Scaled Group Lasso
• Inference after black box selection
• Statistical Inference For Noisy Matrix Completion Incorporating Auxiliary Information
• Estimator Augmentation with Applications in High-Dimensional Group Inference
• Moderate-Dimensional Inferences on Quadratic Functionals in Ordinary Least Squares
• Chi-square and normal inference in high-dimensional multi-task regression
• Divide-and-conquer methods for big data analysis
• Confidence intervals for parameters in high-dimensional sparse vector autoregression
• Non-asymptotic error controlled sparse high dimensional precision matrix estimation
• Single Point Transductive Prediction
• Inference in High-Dimensional Linear Measurement Error Models
• Uniformly valid confidence intervals for conditional treatment effects in misspecified high-dimensional models
• Optimal estimation of functionals of high-dimensional mean and covariance matrix
• Markov Neighborhood Regression for High-Dimensional Inference
• Sketching in Bayesian High Dimensional Regression With Big Data Using Gaussian Scale Mixture Priors
• Integrative Factor Regression and Its Inference for Multimodal Data Analysis
• Residual permutation test for regression coefficient testing
• Asymptotic normality of robust $M$-estimators with convex penalty
• Uncertainty Quantification for Demand Prediction in Contextual Dynamic Pricing
• Inference for High Dimensional Censored Quantile Regression
• Statistical Inference in High-dimensional Generalized Linear Models with Streaming Data
• High-Dimensional Semiparametric Selection Models: Estimation Theory with an Application to the Retail Gasoline Market
• Variable Selection via Adaptive False Negative Control in Linear Regression
• Statistical Inference for High-Dimensional Linear Regression with Blockwise Missing Data
• Inference for multiple treatment effects using confounder importance learning
• Robust High-Dimensional Time-Varying Coefficient Estimation
• Semi-supervised linear regression: enhancing efficiency and robustness in high dimensions
• Surrogate Aided Unsupervised Recovery of Sparse Signals in Single Index Models for Binary Outcomes
• Statistical significance in high-dimensional linear mixed models
• Asymptotic normality in linear regression with approximately sparse structure
• Robust Inference for High-Dimensional Linear Models via Residual Randomization
• On the Properties of Simulation-based Estimators in High Dimensions
• Inference robust to outliers with l1-norm penalization
• Kernel Ordinary Differential Equations
• Statistical inference and feasibility determination: a nonasymptotic approach
• Worst possible sub-directions in high-dimensional models
• Causal Inference with Invalid Instruments: Post-selection Problems and A Solution Using Searching and Sampling
• Bias Correction for Regularized Regression and its Application in Learning with Streaming Data
• Testing Endogeneity with High Dimensional Covariates
• Simultaneous Estimation of Multiple Treatment Effects from Observational Studies
• Understanding the population structure correction regression
• Scalable simultaneous inference in high-dimensional linear regression models
• The Adaptive Multi-Factor Model and the Financial Market
• Measurement error and reliability from a different perspective: Reliability of decision functions and the sum score
• High-dimensional inference robust to outliers with l1-norm penalization
• Logistic regression and Ising networks: prediction and estimation when violating lasso assumptions
• StarTrek: Combinatorial Variable Selection with False Discovery Rate Control
• High-Dimensional Inference Based on the Leave-One-Covariate-Out LASSO Path
• Multivariate Log-Contrast Regression with Sub-Compositional Predictors: Testing the Association Between Preterm Infants' Gut Microbiome and Neurobehavioral Outcomes
• Inference Without Compatibility
• Simultaneous Parameter Learning and Bi-Clustering for Multi-Response Models