Nearly unbiased variable selection under minimax concave penalty

Abstract

We propose MC+, a fast, continuous, nearly unbiased and accurate method of penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO may prevent consistent variable selection. Subset selection is unbiased but computationally costly. The MC+ has two elements: a minimax concave penalty (MCP) and a penalized linear unbiased selection (PLUS) algorithm. The MCP provides the convexity of the penalized loss in sparse regions to the greatest extent given certain thresholds for variable selection and unbiasedness. The PLUS computes multiple exact local minimizers of a possibly nonconvex penalized loss function in a certain main branch of the graph of critical points of the penalized loss. Its output is a continuous piecewise linear path encompassing from the origin for infinite penalty to a least squares solution for zero penalty. We prove that at a universal penalty level, the MC+ has high probability of matching the signs of the unknowns, and thus correct selection, without assuming the strong irrepresentable condition required by the LASSO. This selection consistency applies to the case of $p\gg n$, and is proved to hold for exactly the MC+ solution among possibly many local minimizers. We prove that the MC+ attains certain minimax convergence rates in probability for the estimation of regression coefficients in ${\ell }_r$ balls. We use the SURE method to derive degrees of freedom and $C_p$-type risk estimates for general penalized LSE, including the LASSO and MC+ estimators, and prove their unbiasedness. Based on the estimated degrees of freedom, we propose an estimator of the noise level for proper choice of the penalty level.

Published in at http://dx.doi.org/10.1214/09-AOS729 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (26)

• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Rejoinder to "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• High-dimensional graphs and variable selection with the Lasso
• Nonconcave penalized likelihood with a diverging number of parameters
• One-step sparse estimates in nonconcave penalized likelihood models
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Piecewise linear regularized solution paths
• Lasso-type recovery of sparse representations for high-dimensional data
• High-dimensional generalized linear models and the lasso
• Variable selection using MM algorithms
• Sparsity oracle inequalities for the Lasso
• Sharp thresholds for high-dimensional and noisy recovery of sparsity
• Discussion: The Dantzig selector: Statistical estimation when $p$ is much larger than $n$
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• Discussion: A tale of three cousins: Lasso, L2Boosting and Dantzig
• Bolasso: model consistent Lasso estimation through the bootstrap
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models

Cited by in corpus (616)

• Challenges of Big Data Analysis
• A survey of sparse representation: algorithms and applications
• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
• Controlling the false discovery rate via knockoffs
• Variable selection in nonparametric additive models
• MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
• Sparse Regularization via Convex Analysis
• The Emerging Trends of Multi-Label Learning
• Model Selection Techniques -- An Overview
• Regularized estimation in sparse high-dimensional time series models
• Strong oracle optimality of folded concave penalized estimation
• Nonconvex Nonsmooth Low-Rank Minimization via Iteratively Reweighted Nuclear Norm
• A Selective Review of Group Selection in High-Dimensional Models
• Generalized Nonconvex Nonsmooth Low-Rank Minimization
• A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems
• Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors
• Bayesian linear regression with sparse priors
• Tuning parameter selection in high dimensional penalized likelihood
• Regularized Matrix Regression
• Statistical significance in high-dimensional linear models
• Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods
• L1-Penalization for Mixture Regression Models
• Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima
• High-Dimensional Inference: Confidence Intervals, $p$-Values and R-Software hdi
• Extended Comparisons of Best Subset Selection, Forward Stepwise Selection, and the Lasso
• Adaptive robust variable selection
• Partially linear additive quantile regression in ultra-high dimension
• Regularization for Cox's proportional hazards model with NP-dimensionality
• Robust PCA via Nonconvex Rank Approximation
• Calibrating nonconvex penalized regression in ultra-high dimension
• The fused Kolmogorov filter: A nonparametric model-free screening method
• Total Variation Denoising via the Moreau Envelope
• Feature Selection with Annealing for Computer Vision and Big Data Learning
• Endogeneity in high dimensions
• Sparse Regression Learning by Aggregation and Langevin Monte-Carlo
• Variable selection in linear mixed effects models
• Enhanced Low-Rank Matrix Approximation
• Non-convex Regularizations for Feature Selection in Ranking With Sparse SVM
• ${\ell }_0$-Motivated Low-Rank Sparse Subspace Clustering
• High-dimensional variable selection via tilting
• Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
• The sparse Laplacian shrinkage estimator for high-dimensional regression
• Exponential Screening and optimal rates of sparse estimation
• Regularization Methods for High-Dimensional Instrumental Variables Regression With an Application to Genetical Genomics
• High-Dimensional Sparse Additive Hazards Regression
• Simultaneous Variable and Covariance Selection with the Multivariate Spike-and-Slab Lasso
• Sparse Matrix Inversion with Scaled Lasso
• Multiple testing of local maxima for detection of peaks in 1D
• High dimensional generalized empirical likelihood for moment restrictions with dependent data
• What Objective Does Self-paced Learning Indeed Optimize?
• An adaptive Ridge procedure for L0 regularization
• Sparse Regression: Scalable algorithms and empirical performance
• Distributed Estimation and Inference with Statistical Guarantees
• Innovated interaction screening for high-dimensional nonlinear classification
• Least Squares Approximation for a Distributed System
• Mind the duality gap: safer rules for the Lasso
• Sparse Estimation by Exponential Weighting
• A Tight Bound of Hard Thresholding
• Linear Convergence of Adaptively Iterative Thresholding Algorithms for Compressed Sensing
• A Practical Scheme and Fast Algorithm to Tune the Lasso With Optimality Guarantees
• Generalized Singular Value Thresholding
• Nonconcave penalized composite conditional likelihood estimation of sparse Ising models
• Nearly optimal Bayesian Shrinkage for High Dimensional Regression
• High dimensional thresholded regression and shrinkage effect
• Goodness of fit tests for high-dimensional linear models
• Learning Large-Scale Bayesian Networks with the sparsebn Package
• Concave Penalized Estimation of Sparse Gaussian Bayesian Networks
• Linearly Involved Generalized Moreau Enhanced Models and Their Proximal Splitting Algorithm under Overall Convexity Condition
• Ridge regression and asymptotic minimax estimation over spheres of growing dimension
• Covariate assisted screening and estimation
• Only Train Once: A One-Shot Neural Network Training And Pruning Framework
• Spike-and-Slab Meets LASSO: A Review of the Spike-and-Slab LASSO
• A new scope of penalized empirical likelihood with high-dimensional estimating equations
• Shrinkage Estimation Strategies in Generalized Ridge Regression Models Under Low/High-Dimension Regime
• Log-ratio Lasso: Scalable, Sparse Estimation for Log-ratio Models
• Successive Concave Sparsity Approximation for Compressed Sensing
• Are Discoveries Spurious? Distributions of Maximum Spurious Correlations and Their Applications
• Nested Sparse Approximation: Structured Estimation of V2V Channels Using Geometry-Based Stochastic Channel Model
• A majorization-minimization approach to variable selection using spike and slab priors
• Tight conditions for consistency of variable selection in the context of high dimensionality
• Nonconvex Sparse Logistic Regression with Weakly Convex Regularization
• Vector-Valued Graph Trend Filtering with Non-Convex Penalties
• Optimization Methods for Convolutional Sparse Coding
• High-Dimensional Bayesian Regularised Regression with the BayesReg Package
• Asymptotic equivalence of regularization methods in thresholded parameter space
• Optimizing regularized Cholesky score for order-based learning of Bayesian networks
• Sparse Classification: a scalable discrete optimization perspective
• A globally convergent algorithm for nonconvex optimization based on block coordinate update
• A Shrinkage Principle for Heavy-Tailed Data: High-Dimensional Robust Low-Rank Matrix Recovery
• Global solutions to folded concave penalized nonconvex learning
• Convex and non-convex regularization methods for spatial point processes intensity estimation
• Nonconvex penalties with analytical solutions for one-bit compressive sensing
• Learning Directed Acyclic Graphs with Penalized Neighbourhood Regression
• Robust and Sparse Regression via $\gamma$-divergence
• Distributed High-dimensional Regression Under a Quantile Loss Function
• A Novel Regularization Based on the Error Function for Sparse Recovery
• Variance prior forms for high-dimensional Bayesian variable selection
• Nonconvex Sparse Learning via Stochastic Optimization with Progressive Variance Reduction
• Non-convex regularization in remote sensing
• Model selection and structure specification in ultra-high dimensional generalised semi-varying coefficient models
• Asymptotic properties for combined $L_1$ and concave regularization
• Scalar on time-by-distribution regression and its application for modelling associations between daily-living physical activity and cognitive functions in Alzheimer's Disease
• Sparse and Functional Principal Components Analysis
• Distributed recovery of jointly sparse signals under communication constraints
• A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models
• Scalable Bayesian Variable Selection Using Nonlocal Prior Densities in Ultrahigh-Dimensional Settings
• Implicit differentiation of Lasso-type models for hyperparameter optimization
• Penalized estimation in high-dimensional hidden Markov models with state-specific graphical models
• Minimizing L1 over L2 norms on the gradient
• Estimation of Optimal Individualized Treatment Rules Using a Covariate-Specific Treatment Effect Curve with High-dimensional Covariates
• ATASI-Net: An Efficient Sparse Reconstruction Network for Tomographic SAR Imaging with Adaptive Threshold
• Dynamic Visualization and Fast Computation for Convex Clustering via Algorithmic Regularization
• Regularization vs. Relaxation: A conic optimization perspective of statistical variable selection
• Rate Optimal Denoising of Simultaneously Sparse and Low Rank Matrices
• Doubly-Robust Lasso Bandit
• MM Algorithms for Minimizing Nonsmoothly Penalized Objective Functions
• Efficient DC Algorithm for Constrained Sparse Optimization
• Structured Nonconvex and Nonsmooth Optimization: Algorithms and Iteration Complexity Analysis
• Adaptive LASSO estimation for functional hidden dynamic geostatistical model
• Ultrahigh Dimensional Variable Selection for Mapping Soil Carbon
• A general theory of regression adjustment for covariate-adaptive randomization: OLS, Lasso, and beyond
• Sparse PCA with Oracle Property
• Linearly-involved Moreau-Enhanced-over-Subspace Model: Debiased Sparse Modeling and Stable Outlier-Robust Regression
• Solution Path Clustering with Adaptive Concave Penalty
• Rank-one Convexification for Sparse Regression
• Iterative Soft/Hard Thresholding with Homotopy Continuation for Sparse Recovery
• Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality
• Tucker Tensor Regression and Neuroimaging Analysis
• Expandable Factor Analysis
• Preconditioning to comply with the Irrepresentable Condition
• Minimization of Transformed $L_1$ Penalty: Theory, Difference of Convex Function Algorithm, and Robust Application in Compressed Sensing
• De-Biasing The Lasso With Degrees-of-Freedom Adjustment
• Generalized Linear Model Regression under Distance-to-set Penalties
• Interaction Pursuit with Feature Screening and Selection
• Estimation And Selection Via Absolute Penalized Convex Minimization And Its Multistage Adaptive Applications
• Scaled Sparse Linear Regression
• Robust Estimation of High-Dimensional Mean Regression
• High-dimensional empirical likelihood inference
• Sparse and Smooth Signal Estimation: Convexification of L0 Formulations
• A review of regularised estimation methods and cross-validation in spatiotemporal statistics
• Tensor Recovery Based on A Novel Non-convex Function Minimax Logarithmic Concave Penalty Function
• False Discoveries Occur Early on the Lasso Path
• RaSE: A Variable Screening Framework via Random Subspace Ensembles
• Item Response Theory -- A Statistical Framework for Educational and Psychological Measurement
• Ultrahigh-dimensional Robust and Efficient Sparse Regression using Non-Concave Penalized Density Power Divergence
• Mixed-Projection Conic Optimization: A New Paradigm for Modeling Rank Constraints
• Fast Low-Rank Matrix Learning with Nonconvex Regularization
• Uniform Inference for High-dimensional Quantile Regression: Linear Functionals and Regression Rank Scores
• Fully Bayesian Logistic Regression with Hyper-Lasso Priors for High-dimensional Feature Selection
• Doubly Robust Sure Screening for Elliptical Copula Regression Model
• On the Beta Prime Prior for Scale Parameters in High-Dimensional Bayesian Regression Models
• A novel algorithm for simultaneous SNP selection in high-dimensional genome-wide association studies
• Penalized Estimation of Frailty-Based Illness-Death Models for Semi-Competing Risks
• Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation
• Fully Bayesian Classification with Heavy-tailed Priors for Selection in High-dimensional Features with Grouping Structure
• Simultaneous Feature Selection and Outlier Detection with Optimality Guarantees
• Designing Patient-Specific Optimal Neurostimulation Patterns for Seizure Suppression
• Variable Smoothing for Weakly Convex Composite Functions
• Outlyingness: why do outliers lie out?
• An unbiased approach to compressed sensing
• A ROAD to Classification in High Dimensional Space
• On optimal solutions of the constrained ${\ell }_0$ regularization and its penalty problem
• Stability selection enables robust learning of partial differential equations from limited noisy data
• Distributed Proximal Gradient Algorithm for Partially Asynchronous Computer Clusters
• Cross validation approaches for penalized Cox regression
• High-Dimensional Linear Regression via Implicit Regularization
• Array SAR 3D Sparse Imaging Based on Regularization by Denoising Under Few Observed Data
• Selection of tuning parameters in bridge regression models via Bayesian information criterion
• Strong NP-Hardness for Sparse Optimization with Concave Penalty Functions
• Nonconvex Optimization via MM Algorithms: Convergence Theory
• Nonconvex and Nonsmooth Sparse Optimization via Adaptively Iterative Reweighted Methods
• Tight conditions for consistent variable selection in high dimensional nonparametric regression
• Confidence Intervals for Low-Dimensional Parameters in High-Dimensional Linear Models
• DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measures
• Combined l_1 and greedy l_0 penalized least squares for linear model selection
• Scalable Algorithms for Large Competing Risks Data
• The Trimmed Lasso: Sparsity and Robustness
• Greedy algorithms for prediction
• Schatten-$p$ Quasi-Norm Regularized Matrix Optimization via Iterative Reweighted Singular Value Minimization
• Multi-stage Convex Relaxation for Feature Selection
• Variance Reduced Median-of-Means Estimator for Byzantine-Robust Distributed Inference
• Path Following and Empirical Bayes Model Selection for Sparse Regression
• A Unified Framework for Sparse Relaxed Regularized Regression: SR3
• Towards Faster Rates and Oracle Property for Low-Rank Matrix Estimation
• Robust subset selection
• Supervised clustering of high dimensional data using regularized mixture modeling
• Variable selection and estimation for semi-parametric multiple-index models
• Robustness of Sparse Recovery via $F$-minimization: A Topological Viewpoint
• A Unified Framework for Solving a General Class of Nonconvexly Regularized Convex Models
• High-Dimensional Composite Quantile Regression: Optimal Statistical Guarantees and Fast Algorithms
• Robustness in sparse linear models: relative efficiency based on robust approximate message passing
• Second order Stein: SURE for SURE and other applications in high-dimensional inference
• Enhanced total variation minimization for stable image reconstruction
• Ranked Sparsity: A Cogent Regularization Framework for Selecting and Estimating Feature Interactions and Polynomials
• SLOPE is Adaptive to Unknown Sparsity and Asymptotically Minimax
• regsem: Regularized Structural Equation Modeling
• The Backbone Method for Ultra-High Dimensional Sparse Machine Learning
• Adaptive Minimax Estimation over Sparse ${\ell }_q$-Hulls
• On Nonconvex Decentralized Gradient Descent
• Network reconstruction via the minimum description length principle
• High-dimensional Censored Regression via the Penalized Tobit Likelihood
• Screening Rules for Lasso with Non-Convex Sparse Regularizers
• Alternating Direction Method of Multipliers for A Class of Nonconvex and Nonsmooth Problems with Applications to Background/Foreground Extraction
• Multiple-Splitting Projection Test for High-Dimensional Mean Vectors
• DIF Analysis with Unknown Groups and Anchor Items
• The adaptive and the thresholded Lasso for potentially misspecified models
• Statistical Inferences Using Large Estimated Covariances for Panel Data and Factor Models
• Unified Scalable Equivalent Formulations for Schatten Quasi-Norms
• Scalable Sparse Cox's Regression for Large-Scale Survival Data via Broken Adaptive Ridge
• Learning Sparse Classifiers: Continuous and Mixed Integer Optimization Perspectives
• Penalized integrative analysis under the accelerated failure time model
• Pathwise Coordinate Optimization for Sparse Learning: Algorithm and Theory
• Selective Factor Extraction in High Dimensions
• Main and Interaction Effects Selection for Quadratic Discriminant Analysis via Penalized Linear Regression
• Tensor Generalized Estimating Equations for Longitudinal Imaging Analysis
• The springback penalty for robust signal recovery
• Between hard and soft thresholding: optimal iterative thresholding algorithms
• A Likelihood Ratio Framework for High Dimensional Semiparametric Regression
• An Overview on the Estimation of Large Covariance and Precision Matrices
• Proximal Gradient Method with Extrapolation and Line Search for a Class of Nonconvex and Nonsmooth Problems
• On the Computational Complexity of High-Dimensional Bayesian Variable Selection
• The restricted consistency property of leave-$n_v$-out cross-validation for high-dimensional variable selection
• Distributional Results for Thresholding Estimators in High-Dimensional Gaussian Regression Models
• I-LAMM for Sparse Learning: Simultaneous Control of Algorithmic Complexity and Statistical Error
• Certifiably Optimal Sparse Inverse Covariance Estimation
• Reduced-rank Regression in Sparse Multivariate Varying-Coefficient Models with High-dimensional Covariates
• No penalty no tears: Least squares in high-dimensional linear models
• Flexible covariate-adjusted exact tests of randomized treatment effects with application to a trial of HIV education
• Efficient algorithm to select tuning parameters in sparse regression modeling with regularization
• Parallel integrative learning for large-scale multi-response regression with incomplete outcomes
• FIRE: An Optimization Approach for Fast Interpretable Rule Extraction
• A Unified Framework for Constructing Nonconvex Regularizations
• Sparse Signal Reconstruction for Nonlinear Models via Piecewise Rational Optimization
• Inference for high-dimensional linear mixed-effects models: A quasi-likelihood approach
• Transformed ${\ell }_1$ Regularization for Learning Sparse Deep Neural Networks
• Bayesian inference in high-dimensional linear models using an empirical correlation-adaptive prior
• Simple Stochastic Gradient Methods for Non-Smooth Non-Convex Regularized Optimization
• Universal Stagewise Learning for Non-Convex Problems with Convergence on Averaged Solutions
• A Novel Framework for Online Supervised Learning with Feature Selection
• A Scalable Empirical Bayes Approach to Variable Selection in Generalized Linear Models
• Inference for high-dimensional instrumental variables regression
• Large-Scale Low-Rank Matrix Learning with Nonconvex Regularizers
• Exploration of heterogeneous treatment effects via concave fusion
• Sparse principal component regression via singular value decomposition approach
• A Survey of Tuning Parameter Selection for High-dimensional Regression
• Horseshoe Regularization for Feature Subset Selection
• Strong rules for nonconvex penalties and their implications for efficient algorithms in high-dimensional regression
• Individualized Multi-directional Variable Selection
• Nearly optimal minimax estimator for high-dimensional sparse linear regression
• Perturbation techniques for convergence analysis of proximal gradient method and other first-order algorithms via variational analysis
• A sparse semismooth Newton based proximal majorization-minimization algorithm for nonconvex square-root-loss regression problems
• Estimation for bivariate quantile varying coefficient model
• Functional Group Bridge for Simultaneous Regression and Support Estimation
• Sparse estimation via nonconcave penalized likelihood in a factor analysis model
• AIC for Non-concave Penalized Likelihood Method
• A Two-Stage Penalized Least Squares Method for Constructing Large Systems of Structural Equations
• A Moreau-Yosida approximation scheme for a class of high-dimensional posterior distributions
• Approximate message passing for nonconvex sparse regularization with stability and asymptotic analysis
• L0TV: A Sparse Optimization Method for Impulse Noise Image Restoration
• Multi-threshold Change Plane Model: Estimation Theory and Applications in Subgroup Identification
• Big Data Analysis Using Shrinkage Strategies
• Nonparametric Independence Screening in Sparse Ultra-High Dimensional Additive Models
• Quantile universal threshold for model selection
• A proximal subgradient algorithm with extrapolation for structured nonconvex nonsmooth problems
• Efficient Learning with a Family of Nonconvex Regularizers by Redistributing Nonconvexity
• Sparse Variable Selection on High Dimensional Heterogeneous Data with Tree Structured Responses
• Scalable high-dimensional Bayesian varying coefficient models with unknown within-subject covariance
• Sparse Recovery via Partial Regularization: Models, Theory and Algorithms
• Monotone Lipschitz-Gradient Denoiser: Explainability of Operator Regularization Approaches Free From Lipschitz Constant Control
• Nonparametric Screening under Conditional Strictly Convex Loss for Ultrahigh Dimensional Sparse Data
• Ideal formulations for constrained convex optimization problems with indicator variables
• Consistent Second-Order Conic Integer Programming for Learning Bayesian Networks
• Sequential Lasso for feature selection with ultra-high dimensional feature space
• Iterative Hard Thresholding for Model Selection in Genome-Wide Association Studies
• Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold
• Sparse Model Identification and Learning for Ultra-high-dimensional Additive Partially Linear Models
• Reducing overestimating and underestimating volatility via the augmented blending-ARCH model
• Penalized pairwise pseudo likelihood for variable selection with nonignorable missing data
• Gradient descent with nonconvex constraints: local concavity determines convergence
• Optimal Estimation and Prediction for Dense Signals in High-Dimensional Linear Models
• Accelerated Stochastic Algorithms for Nonconvex Finite-sum and Multi-block Optimization
• Generalized High-Dimensional Trace Regression via Nuclear Norm Regularization
• Variance function estimation in high-dimensions
• Spectral Analysis of High-dimensional Time Series
• DuRIN: A Deep-unfolded Sparse Seismic Reflectivity Inversion Network
• Minimization of Transformed $L_1$ Penalty: Closed Form Representation and Iterative Thresholding Algorithms
• A News-based Machine Learning Model for Adaptive Asset Pricing
• On Hodges' Superefficiency and Merits of Oracle Property in Model Selection
• Sparsity-Agnostic Lasso Bandit
• Fast Signal Recovery from Saturated Measurements by Linear Loss and Nonconvex Penalties
• Empirical Bayes, SURE and Sparse Normal Mean Models
• Dynamic Sasvi: Strong Safe Screening for Norm-Regularized Least Squares
• Penalized quasi likelihood estimation for variable selection
• High-dimensional robust approximated M-estimators for mean regression with asymmetric data
• Bridging between soft and hard thresholding by scaling
• AcSel: selecting variables with accuracy in correlated datasets
• Katalyst: Boosting Convex Katayusha for Non-Convex Problems with a Large Condition Number
• Cross validation in sparse linear regression with piecewise continuous nonconvex penalties and its acceleration
• Penalized Sparse Covariance Regression with High Dimensional Covariates
• Comparisons of penalized least squares methods by simulations
• Discussion: "A significance test for the lasso"
• On the consistency theory of high dimensional variable screening
• selectBoost: a general algorithm to enhance the performance of variable selection methods in correlated datasets
• Sparse Regularization: Convergence Of Iterative Jumping Thresholding Algorithm
• Semi-Penalized Inference with Direct False Discovery Rate Control in High-Dimensions
• Regularization after retention in ultrahigh dimensional linear regression models
• Semiparametric Sparse Discriminant Analysis
• Regularized Nonlinear Regression for Simultaneously Selecting and Estimating Key Model Parameters
• Nearly Optimal Variational Inference for High Dimensional Regression with Shrinkage Priors
• Thresholded Adaptive Validation: Tuning the Graphical Lasso for Graph Recovery
• Tensor Graphical Model: Non-convex Optimization and Statistical Inference
• SparseStep: Approximating the Counting Norm for Sparse Regularization
• Extended Stochastic Gradient MCMC for Large-Scale Bayesian Variable Selection
• Doubly-robust evaluation of high-dimensional surrogate markers
• A Convex-Nonconvex Strategy for Grouped Variable Selection
• A Sharper Computational Tool for ${\text{L}}_2\text{E}$ Regression
• Variable Selection with Exponential Weights and $l_0$-Penalization
• A High-dimensional M-estimator Framework for Bi-level Variable Selection
• A General Theory of Concave Regularization for High Dimensional Sparse Estimation Problems
• A Sparse Graph-Structured Lasso Mixed Model for Genetic Association with Confounding Correction
• A Survey of Estimation Methods for Sparse High-dimensional Time Series Models
• Improving COVID-19 Forecasting using eXogenous Variables
• Estimation of the genetic Gaussian network using GWAS summary data
• REMI: Regression with marginal information and its application in genome-wide association studies
• On ${\ell }_p$-hyperparameter Learning via Bilevel Nonsmooth Optimization
• Sparse and Low-bias Estimation of High Dimensional Vector Autoregressive Models
• A Feasible Level Proximal Point Method for Nonconvex Sparse Constrained Optimization
• Partial Correlation Graphical LASSO
• Fast algorithms for robust principal component analysis with an upper bound on the rank
• Best subset selection is robust against design dependence
• Culling the herd of moments with penalized empirical likelihood
• Nonconvex sparse regularization for deep neural networks and its optimality
• Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processes
• MM for Penalized Estimation
• Phase transition in compressed sensing with horseshoe prior
• Regularized deep learning with nonconvex penalties
• A Laplace Mixture Representation of the Horseshoe and Some Implications
• ${\ell }_1$-penalized Multinomial Regression: Estimation, inference, and prediction, with an application to risk factor identification for different dementia subtypes
• Revisiting Subgradient Method: Complexity and Convergence Beyond Lipschitz Continuity
• Nonconvex Approach for Sparse and Low-Rank Constrained Models with Dual Momentum
• Spatial Heterogeneity Automatic Detection and Estimation
• Optimal Statistical Inference for Individualized Treatment Effects in High-dimensional Models
• Continuous Relaxation of Discontinuous Shrinkage Operator: Proximal Inclusion and Conversion
• Optimal False Discovery Control of Minimax Estimator
• Non-convex Lasso-kind approach to compressed sensing for finite-valued signals
• An efficient ADMM algorithm for high dimensional precision matrix estimation via penalized quadratic loss
• Online Debiased Lasso for Streaming Data
• ADMM-MCP Framework for Sparse Recovery with Global Convergence
• Graphical Models for Non-Negative Data Using Generalized Score Matching
• Learning Latent Features with Pairwise Penalties in Low-Rank Matrix Completion
• Estimator of Prediction Error Based on Approximate Message Passing for Penalized Linear Regression
• Matrix Completion with Nonconvex Regularization: Spectral Operators and Scalable Algorithms
• A Discussion on Practical Considerations with Sparse Regression Methodologies
• A note relating ridge regression and OLS p-values to preconditioned sparse penalized regression
• Expanded Alternating Optimization of Nonconvex Functions with Applications to Matrix Factorization and Penalized Regression
• Conditional regression based on a multivariate zero-inflated logistic normal model for microbiome relative abundance data
• MOCCA: mirrored convex/concave optimization for nonconvex composite functions
• Balancing Statistical and Computational Precision: A General Theory and Applications to Sparse Regression
• A Rank-Corrected Procedure for Matrix Completion with Fixed Basis Coefficients
• Top-N Recommendation with Novel Rank Approximation
• Efficient Regularized Regression for Variable Selection with L0 Penalty
• The Benefit of Group Sparsity in Group Inference with De-biased Scaled Group Lasso
• Model Selection for High Dimensional Quadratic Regression via Regularization
• Semiparametric Expectile Regression for High-dimensional Heavy-tailed and Heterogeneous Data
• Subsampling Winner Algorithm for Feature Selection in Large Regression Data
• Structure Adaptive Elastic-Net
• Divide-and-conquer methods for big data analysis
• Marginal false discovery rates for penalized regression models
• Markov Neighborhood Regression for High-Dimensional Inference
• On Data Enriched Logistic Regression
• Simple structure estimation via prenet penalization
• Double Sparsity Kernel Learning with Automatic Variable Selection and Data Extraction
• High-Dimensional Expected Shortfall Regression
• Homotopy Continuation Approaches for Robust SV Classification and Regression
• Kinetic Energy Plus Penalty Functions for Sparse Estimation
• Likelihood Adaptively Modified Penalties
• Sparse Fisher's Linear Discriminant Analysis for Partially Labeled Data
• Robust Reduced Rank Regression
• Innovated scalable efficient estimation in ultra-large Gaussian graphical models
• Sketching in Bayesian High Dimensional Regression With Big Data Using Gaussian Scale Mixture Priors
• Partial Penalized Likelihood Ratio Test under Sparse Case
• Composite Difference-Max Programs for Modern Statistical Estimation Problems
• Learning Graph Laplacian with MCP
• Understanding Notions of Stationarity in Non-Smooth Optimization
• Doubly Robust Feature Selection with Mean and Variance Outlier Detection and Oracle Properties
• Uncertainty Quantification Under Group Sparsity
• A variable smoothing for Nonconvexly constrained nonsmooth optimization with application to sparse spectral clustering
• The Sparse Multivariate Method of Simulated Quantiles
• Mixed Effect Modeling and Variable Selection for Quantile Regression
• LOCUS: A Novel Decomposition Method for Brain Network Connectivity Matrices using Low-rank Structure with Uniform Sparsity
• Linear time dynamic programming for the exact path of optimal models selected from a finite set
• Minimization of the $q$-ratio sparsity with $1<q\le \infty$ for signal recovery
• Regularization Methods Based on the $L_q$-Likelihood for Linear Models with Heavy-Tailed Errors
• Penalty methods for a class of non-Lipschitz optimization problems
• Bayesian Variable Selection for Skewed Heteroscedastic Response
• Statistical inference for high dimensional regression via Constrained Lasso
• Integrative Factor Regression and Its Inference for Multimodal Data Analysis
• A Novel Approach for Fast Detection of Multiple Change Points in Linear Models
• Deconfounding and Causal Regularization for Stability and External Validity
• A refined convergence analysis of pDCA${}_e$ with applications to simultaneous sparse recovery and outlier detection
• FasTer: Fast Tensor Completion with Nonconvex Regularization
• Optimization Landscapes of Wide Deep Neural Networks Are Benign
• Data-Driven Logistic Regression Ensembles With Applications in Genomics
• A sparse negative binomial mixture model for clustering RNA-seq count data
• Phase transition in binary compressed sensing based on $L_1$-norm minimization
• A Variational Approach on Level sets and Linear Convergence of Variable Bregman Proximal Gradient Method for Nonconvex Optimization Problems
• An adaptive shortest-solution guided decimation approach to sparse high-dimensional linear regression
• Optimal estimation of functionals of high-dimensional mean and covariance matrix
• SNAP: A semismooth Newton algorithm for pathwise optimization with optimal local convergence rate and oracle properties
• A Unified Convex Surrogate for the Schatten-$p$ Norm
• Penalized robust estimators in logistic regression with applications to sparse models
• Low-Rank and Total Variation Regularization and Its Application to Image Recovery
• Bayesian high-dimensional linear regression with generic spike-and-slab priors
• Scaled minimax optimality in high-dimensional linear regression: A non-convex algorithmic regularization approach
• Structured Sparsity Inducing Adaptive Optimizers for Deep Learning
• Nonconvex penalized multitask regression using data depth-based penalties
• Smoothing the Edges: Smooth Optimization for Sparse Regularization using Hadamard Overparametrization
• SG-PALM: a Fast Physically Interpretable Tensor Graphical Model
• Model Selection Confidence Sets by Likelihood Ratio Testing
• Matrix Completion via Nonconvex Regularization: Convergence of the Proximal Gradient Algorithm
• Sorted Concave Penalized Regression
• Low-rank matrix estimation in multi-response regression with measurement errors: Statistical and computational guarantees
• Robust Variable Selection Criteria for the Penalized Regression
• NuSPAN: A Proximal Average Network for Nonuniform Sparse Model -- Application to Seismic Reflectivity Inversion
• Distribution Regression
• Network induced large correlation matrix estimation
• Accelerated Gradient Methods for Sparse Statistical Learning with Nonconvex Penalties
• Approximate Laplace approximations for scalable model selection
• Improving Network Slimming with Nonconvex Regularization
• Stochastic Optimization for DC Functions and Non-smooth Non-convex Regularizers with Non-asymptotic Convergence
• Macroeconomic Forecasting and Variable Selection with a Very Large Number of Predictors: A Penalized Regression Approach
• Simultaneous Heterogeneity and Reduced-rank Learning for Multivariate Response Regression
• Bilinear Factor Matrix Norm Minimization for Robust PCA: Algorithms and Applications
• Discussion: "A significance test for the lasso"
• Variable Selection Using Relative Importance Rankings
• Structured Sparsity Promoting Functions
• Honest confidence sets for high-dimensional regression by projection and shrinkage
• An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines
• A Proximal Variable Smoothing for Nonsmooth Minimization Involving Weakly Convex Composite with MIMO Application
• Estimation of oblique structure via penalized likelihood factor analysis
• An Optimization Framework with Flexible Inexact Inner Iterations for Nonconvex and Nonsmooth Programming
• A Survey on Nonconvex Regularization Based Sparse and Low-Rank Recovery in Signal Processing, Statistics, and Machine Learning
• T-SVD Based Non-convex Tensor Completion and Robust Principal Component Analysis
• Sparse and Robust Linear Regression: An Optimization Algorithm and Its Statistical Properties
• Scalable Algorithms for the Sparse Ridge Regression
• Higher Criticism Tuned Regression For Weak And Sparse Signals
• Integrated path stability selection
• A Prediction Divergence Criterion for Model Selection
• Stochastic Gradient Descent for Stochastic Doubly-Nonconvex Composite Optimization
• M-estimation with the Trimmed l1 Penalty
• High-dimensional Adaptive Minimax Sparse Estimation with Interactions
• Lockout: Sparse Regularization of Neural Networks
• Calibrated zero-norm regularized LS estimator for high-dimensional error-in-variables regression
• A Provable Smoothing Approach for High Dimensional Generalized Regression with Applications in Genomics
• An Imputation-Consistency Algorithm for High-Dimensional Missing Data Problems and Beyond
• Statistical Inference in High-dimensional Generalized Linear Models with Streaming Data
• A Semi-Smooth Newton Algorithm for High-Dimensional Nonconvex Sparse Learning
• High-Dimensional Semiparametric Selection Models: Estimation Theory with an Application to the Retail Gasoline Market
• Penalized Likelihood Estimation in High-Dimensional Time Series Models and its Application
• High-Dimensional Feature Selection for Sample Efficient Treatment Effect Estimation
• Structural Change in Sparsity
• On the Power of Localized Perceptron for Label-Optimal Learning of Halfspaces with Adversarial Noise
• Sparsity Oriented Importance Learning for High-dimensional Linear Regression
• The scalable Birth-Death MCMC Algorithm for Mixed Graphical Model Learning with Application to Genomic Data Integration
• Integrative Sparse Partial Least Squares
• Sparse recovery via nonconvex regularized $M$-estimators over ${\ell }_q$-balls
• Modeling Between-Study Heterogeneity for Improved Reproducibility in Gene Signature Selection and Clinical Prediction
• Oracle inequalities for sign constrained generalized linear models
• One-Bit Compressed Sensing via One-Shot Hard Thresholding
• On Quadratic Convergence of DC Proximal Newton Algorithm for Nonconvex Sparse Learning in High Dimensions
• A robust Bayesian analysis of variable selection under prior ignorance
• SOFAR: large-scale association network learning
• Clustering of longitudinal curves via a penalized method and EM algorithm
• Local optimization-based statistical inference
• Estimation and inference for high-dimensional non-sparse models
• Global Adaptive Generative Adjustment
• A novel nonconvex, smooth-at-origin penalty for statistical learning
• Optimal prediction for sparse linear models? Lower bounds for coordinate-separable M-estimators
• Gene-Environment Interaction: A Variable Selection Perspective
• A proximal difference-of-convex algorithm with extrapolation
• Generalization error minimization: a new approach to model evaluation and selection with an application to penalized regression
• A New Perspective on Debiasing Linear Regressions
• Accelerated Block Coordinate Proximal Gradients with Applications in High Dimensional Statistics
• A proximal MM method for the zero-norm regularized PLQ composite optimization problem
• Variable Selection with Second-Generation P-Values
• On the superiority of PGMs to PDCAs in nonsmooth nonconvex sparse regression
• Ultrahigh Dimensional Feature Selection via Kernel Canonical Correlation Analysis
• The Flip Side of the Reweighted Coin: Duality of Adaptive Dropout and Regularization
• Relaxed Sparse Eigenvalue Conditions for Sparse Estimation via Non-convex Regularized Regression
• Weak Signal Identification and Inference in Penalized Model Selection
• Variable Selection for Survival Data with A Class of Adaptive Elastic Net Techniques
• Robust Estimation and Shrinkage in Ultrahigh Dimensional Expectile Regression with Heavy Tails and Variance Heterogeneity
• Adaptive Threshold Estimation by FDR
• A constrained L1 minimization approach for estimating multiple Sparse Gaussian or Nonparanormal Graphical Models
• IFAA: Robust association identification and Inference For Absolute Abundance in microbiome analyses
• A Simple Correction Procedure for High-Dimensional Generalized Linear Models with Measurement Error
• Post Selection Shrinkage Estimation for High Dimensional Data Analysis
• A New Nonconvex Strategy to Affine Matrix Rank Minimization Problem
• Sharp RIP Bound for Sparse Signal and Low-Rank Matrix Recovery
• Inference for biased models: a quasi-instrumental variable approach
• Promoting Similarity of Sparsity Structures in Integrative Analysis with Penalization
• Mean and variance estimation in high-dimensional heteroscedastic models with non-convex penalties
• A Greedy Homotopy Method for Regression with Nonconvex Constraints
• Forward variable selection for sparse ultra-high dimensional varying coefficient models
• Efficient and Robust Recovery of Sparse Signal and Image Using Generalized Nonconvex Regularization
• Variable Selection with Scalable Bootstrap in Generalized Linear Model for Massive Data
• Penalized Variable Selection for Multi-center Competing Risks Data
• Capped Lp approximations for the composite L0 regularization problem
• An Iteratively Re-weighted Method for Problems with Sparsity-Inducing Norms
• Linear Convergence of Variable Bregman Stochastic Coordinate Descent Method for Nonsmooth Nonconvex Optimization by Level-set Variational Analysis
• Sparse Optimization Problem with s-difference Regularization
• Robust Model Selection for Finite Mixture of Regression Models Through Trimming
• Smooth Adjustment for Correlated Effects
• Bayesian Fusion Estimation via t-Shrinkage
• ${\ell }_0$-Regularized High-dimensional Accelerated Failure Time Model
• Generalization Bounds for High-dimensional M-estimation under Sparsity Constraint
• A Support Detection and Root Finding Approach for Learning High-dimensional Generalized Linear Models
• Nonconvex Nonsmooth Low-Rank Minimization for Generalized Image Compressed Sensing via Group Sparse Representation
• Renyi Differentially Private ADMM for Non-Smooth Regularized Optimization
• Computing the degrees of freedom of rank-regularized estimators and cousins
• Model detection and variable selection for mode varying coefficient model
• An Easy-to-Implement Hierarchical Standardization for Variable Selection Under Strong Heredity Constraint
• Quantile regression for compositional covariates
• Nested Model Averaging on Solution Path for High-dimensional Linear Regression
• Parallel subgroup analysis of high-dimensional data via M-regression
• A General Framework of Nonparametric Feature Selection in High-Dimensional Data
• A Two-Stage Variable Selection Approach for Correlated High Dimensional Predictors
• A General Framework of Online Updating Variable Selection for Generalized Linear Models with Streaming Datasets
• An Extrapolated Iteratively Reweighted l1 Method with Complexity Analysis
• Quantile index regression
• yaglm: a Python package for fitting and tuning generalized linear models that supports structured, adaptive and non-convex penalties
• Coordinate Descent for MCP/SCAD Penalized Least Squares Converges Linearly
• A provable two-stage algorithm for penalized hazards regression
• Learning Bayesian Networks through Birkhoff Polytope: A Relaxation Method
• A proximal-proximal majorization-minimization algorithm for nonconvex tuning-free robust regression problems
• Pre-processing with Orthogonal Decompositions for High-dimensional Explanatory Variables
• DCA based Algorithm with Extrapolation for Nonconvex Nonsmooth Optimization
• Ultra High Dimensional Change Point Detection
• Joint Learning of Multiple Differential Networks with fMRI data for Brain Connectivity Alteration Detection
• A Majorization Penalty Method for SVM with Sparse Constraint
• Deep Unrolling for Nonconvex Robust Principal Component Analysis
• The Correlation Thresholding Algorithm for Exploratory Factor Analysis
• Adaptive Bi-Level Variable Selection of Conditional Main Effects for Generalized Linear Models
• Variable Selection in Functional Linear Cox Model
• Comparative Review of Modern Competing Risk Methods in High-dimensional Settings
• plmmr: an R package to fit penalized linear mixed models for genome-wide association data with complex correlation structure
• High-dimensional Statistical Inference and Variable Selection Using Sufficient Dimension Association
• A Convex-Nonconvex Framework for Enhancing Minimization Induced Penalties
• KL-BSS: Rethinking optimality for neighbourhood selection in structural equation models
• Cox reduction and confidence sets of models: a theoretical elucidation
• Inexact proximal DC Newton-type method for nonconvex composite functions
• Prior Intensified Information Criterion
• DC algorithms for a class of sparse group ${\ell }_0$ regularized optimization problems
• Targeted Cross-Validation
• Predicting Census Survey Response Rates With Parsimonious Additive Models and Structured Interactions
• Bayesian $L_{\frac{1}{2}}$ regression
• On sure early selection of the best subset
• Neural Optimization Kernel: Towards Robust Deep Learning
• A Convexly Constrained LiGME Model and Its Proximal Splitting Algorithm
• Quantized Proximal Averaging Network for Analysis Sparse Coding
• Nonparametric Quantile Regression for Homogeneity Pursuit in Panel Data Models
• Robust penalized empirical likelihood in high dimensional longitudinal data analysis
• A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning
• Fused Spatial Point Process Intensity Estimation with Varying Coefficients on Complex Constrained Domains
• Two-directional simultaneous inference for high-dimensional models
• Characterization of Excess Risk for Locally Strongly Convex Population Risk
• Effective Proximal Methods for Non-convex Non-smooth Regularized Learning
• Adversarial Robust Low Rank Matrix Estimation: Compressed Sensing and Matrix Completion
• A Scalable, Adaptive and Sound Nonconvex Regularizer for Low-rank Matrix Completion
• HALO: Learning to Prune Neural Networks with Shrinkage
• Testing Mediation Effects Using Logic of Boolean Matrices
• Projection Neural Network for a Class of Sparse Regression Problems with Cardinality Penalty
• A non-convex regularization approach for stable estimation of loss development factors
• Universal sieve-based strategies for efficient estimation using machine learning tools
• Multivariate Temporal Point Process Regression
• Higher Order Refinements by Bootstrap in Lasso and other Penalized Regression Methods
• Algorithmic Versatility of SPF-regularization Methods
• Semiparametric Wavelet-based JPEG IV Estimator for endogenously truncated data
• Iteratively Reweighted ${\ell }_1$-Penalized Robust Regression
• A proximal dual semismooth Newton method for computing zero-norm penalized QR estimator
• From Group Sparse Coding to Rank Minimization: A Novel Denoising Model for Low-level Image Restoration
• KL property of exponent $1/2$ of quadratic functions under nonnegative zero-norm constraints and applications
• ADMM-IDNN: Iteratively Double-reweighted Nuclear Norm Algorithm for Group-prior based Nonconvex Compressed Sensing via ADMM
• Algorithmic Bayesian Group Gibbs Selection
• MSP: A Multi-step Screening Procedure for Sparse Recovery
• HMLasso: Lasso with High Missing Rate
• Prediction of treatment outcome for autism from structure of the brain based on sure independence screening
• On an improvement of LASSO by scaling
• Low Rank Regularization: A Review
• Sparse Multivariate ARCH Models: Finite Sample Properties
• Stochastic collocation methods via minimization of Transformed $L_1$ penalty
• Semiparametric Bayesian Information Criterion for Model Selection in Ultra-high Dimensional Additive Models
• Run-and-Inspect Method for Nonconvex Optimization and Global Optimality Bounds for R-Local Minimizers
• Model Confidence Bounds for Variable Selection
• Errors-in-variables models with dependent measurements
• On the penalized maximum likelihood estimation of high-dimensional approximate factor model
• A Nonconvex Nonsmooth Regularization Method for Compressed Sensing and Low-Rank Matrix Completion
• On the Consistency of the Bias Correction Term of the AIC for the Non-Concave Penalized Likelihood Method
• Sparse Generalized Principal Component Analysis for Large-scale Applications beyond Gaussianity
• Ranking genetic factors related to age-related maculardegeneration by variable selection confidence sets
• Nonconvex Penalization in Sparse Estimation: An Approach Based on the Bernstein Function
• Robust Learning for Optimal Treatment Decision with NP-Dimensionality
• Variable Selection for Additive Partial Linear Quantile Regression with Missing Covariates
• A concave pairwise fusion approach to subgroup analysis
• Trace Pursuit: A General Framework for Model-Free Variable Selection
• Better Solution Principle: A Facet of Concordance between Optimization and Statistics
• Feature Augmentation via Nonparametrics and Selection (FANS) in High Dimensional Classification
• The Bernstein Function: A Unifying Framework of Nonconvex Penalization in Sparse Estimation
• Sparse Solution of Underdetermined Linear Equations via Adaptively Iterative Thresholding
• On the Conditions of Sparse Parameter Estimation via Log-Sum Penalty Regularization
• APPLE: Approximate Path for Penalized Likelihood Estimators
• Towards Ultrahigh Dimensional Feature Selection for Big Data
• Generalized Thresholding and Online Sparsity-Aware Learning in a Union of Subspaces
• High-dimensional regression with unknown variance
• An Iterative Algorithm for Fitting Nonconvex Penalized Generalized Linear Models with Grouped Predictors