An adaptive shortest-solution guided decimation approach to sparse high-dimensional linear regression

Abstract

High-dimensional linear regression model is the most popular statistical model for high-dimensional data, but it is quite a challenging task to achieve a sparse set of regression coefficients. In this paper, we propose a simple heuristic algorithm to construct sparse high-dimensional linear regression models, which is adapted from the shortest solution-guided decimation algorithm and is referred to as ASSD. This algorithm constructs the support of regression coefficients under the guidance of the least-squares solution of the recursively decimated linear equations, and it applies an early-stopping criterion and a second-stage thresholding procedure to refine this support. Our extensive numerical results demonstrate that ASSD outperforms LASSO, vector approximate message passing, and two other representative greedy algorithms in solution accuracy and robustness. ASSD is especially suitable for linear regression problems with highly correlated measurement matrices encountered in real-world applications.

arXiv comment13 pages, 6 figures

References in corpus (20)

• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Rejoinder to "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Nearly unbiased variable selection under minimax concave penalty
• Message Passing Algorithms for Compressed Sensing
• Pathwise coordinate optimization
• A survey of sparse representation: algorithms and applications
• Sure independence screening in generalized linear models with NP-dimensionality
• Expectation-Maximization Gaussian-Mixture Approximate Message Passing
• Efficient High-Dimensional Inference in the Multiple Measurement Vector Problem
• Adaptive Damping and Mean Removal for the Generalized Approximate Message Passing Algorithm
• Sparse Estimation with the Swept Approximated Message-Passing Algorithm
• High-dimensional regression in practice: an empirical study of finite-sample prediction, variable selection and ranking