A statistical mechanics approach to de-biasing and uncertainty estimation in LASSO for random measurements

Abstract

In high-dimensional statistical inference in which the number of parameters to be estimated is larger than that of the holding data, regularized linear estimation techniques are widely used. These techniques have, however, some drawbacks. First, estimators are biased in the sense that their absolute values are shrunk toward zero because of the regularization effect. Second, their statistical properties are difficult to characterize as they are given as numerical solutions to certain optimization problems. In this manuscript, we tackle such problems concerning LASSO, which is a widely used method for sparse linear estimation, when the measurement matrix is regarded as a sample from a rotationally invariant ensemble. We develop a new computationally feasible scheme to construct a de-biased estimator with a confidence interval and conduct hypothesis testing for the null hypothesis that a certain parameter vanishes. It is numerically confirmed that the proposed method successfully de-biases the LASSO estimator and constructs confidence intervals and p-values by experiments for noisy linear measurements.

arXiv comment22 pages, 9 figures

References in corpus (17)

• Discussion of "Least angle regression" by Efron et al
• Rejoinder to "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Simultaneous analysis of Lasso and Dantzig selector
• Message Passing Algorithms for Compressed Sensing
• On asymptotically optimal confidence regions and tests for high-dimensional models
• Replica Field Theory for Deterministic Models (II): A Non-Random Spin Glass with Glassy Behavior
• A typical reconstruction limit of compressed sensing based on Lp-norm minimization
• Tractable approximations for probabilistic models: The adaptive TAP mean field approach
• Analysis of Regularized LS Reconstruction and Random Matrix Ensembles in Compressed Sensing

Cited by in corpus (4)

• Compressed sensing radar detectors under the row-orthogonal design model: a statistical mechanics perspective
• Role of Bootstrap Averaging in Generalized Approximate Message Passing
• Variational Gaussian Approximation in Replica Analysis of Parametric Models
• Replica Analysis for Ensemble Techniques in Variable Selection