Nonconcave penalized likelihood with a diverging number of parameters

Abstract

A class of variable selection procedures for parametric models via nonconcave penalized likelihood was proposed by Fan and Li to simultaneously estimate parameters and select important variables. They demonstrated that this class of procedures has an oracle property when the number of parameters is finite. However, in most model selection problems the number of parameters should be large and grow with the sample size. In this paper some asymptotic properties of the nonconcave penalized likelihood are established for situations in which the number of parameters tends to \infty as the sample size increases. Under regularity conditions we have established an oracle property and the asymptotic normality of the penalized likelihood estimators. Furthermore, the consistency of the sandwich formula of the covariance matrix is demonstrated. Nonconcave penalized likelihood ratio statistics are discussed, and their asymptotic distributions under the null hypothesis are obtained by imposing some mild conditions on the penalty functions.

Cited by in corpus (159)

• Nearly unbiased variable selection under minimax concave penalty
• On the "degrees of freedom" of the lasso
• On the adaptive elastic-net with a diverging number of parameters
• One-step sparse estimates in nonconcave penalized likelihood models
• Variable selection in nonparametric additive models
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• Piecewise linear regularized solution paths
• Sparsistency and rates of convergence in large covariance matrix estimation
• Variable selection using MM algorithms
• Properties of principal component methods for functional and longitudinal data analysis
• Strong oracle optimality of folded concave penalized estimation
• Robust rank correlation based screening
• Network exploration via the adaptive LASSO and SCAD penalties
• A Selective Review of Group Selection in High-Dimensional Models
• Statistical Challenges with High Dimensionality: Feature Selection in Knowledge Discovery
• A unified approach to model selection and sparse recovery using regularized least squares
• Bayesian variable selection with shrinking and diffusing priors
• Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator
• Penalized Composite Quasi-Likelihood for Ultrahigh-Dimensional Variable Selection
• The Dantzig selector: Statistical estimation when $p$ is much larger than $n$
• SCAD-penalized regression in high-dimensional partially linear models
• Adaptive robust variable selection
• Variable selection and regression analysis for graph-structured covariates with an application to genomics
• Coauthorship and Citation Networks for Statisticians
• Calibrating nonconvex penalized regression in ultra-high dimension
• Functional additive regression
• The Lasso for High-Dimensional Regression with a Possible Change-Point
• "Pre-conditioning" for feature selection and regression in high-dimensional problems
• Oracle inequalities for the lasso in the Cox model
• GEE analysis of clustered binary data with diverging number of covariates
• Variable selection in linear mixed effects models
• Nonparametric Density Estimation for High-Dimensional Data - Algorithms and Applications
• Consistent group selection in high-dimensional linear regression
• Rodeo: Sparse, greedy nonparametric regression
• Best subset selection, persistence in high-dimensional statistical learning and optimization under $l_1$ constraint
• A Smooth and Locally Sparse Estimator for Functional Linear Regression via Functional SCAD Penalty
• Estimation and model selection in generalized additive partial linear models for correlated data with diverging number of covariates
• Consistent selection of tuning parameters via variable selection stability
• Dimension reduction based on constrained canonical correlation and variable filtering
• ANOVA for longitudinal data with missing values
• Variable selection in measurement error models
• Penalized maximum likelihood estimation and variable selection in geostatistics
• Efficiency for Regularization Parameter Selection in Penalized Likelihood Estimation of Misspecified Models
• Effective Spectral Unmixing via Robust Representation and Learning-based Sparsity
• Estimation in high-dimensional linear models with deterministic design matrices
• Discussion: The Dantzig selector: Statistical estimation when $p$ is much larger than $n$
• Asymptotic oracle properties of SCAD-penalized least squares estimators
• Variable selection in functional regression models: a review
• A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)
• A penalized empirical likelihood method in high dimensions
• High-dimensional regression in practice: an empirical study of finite-sample prediction, variable selection and ranking
• Asymptotic equivalence of regularization methods in thresholded parameter space
• Smooth Lasso Estimator for the Function-on-Function Linear Regression Model
• Ultrahigh dimensional variable selection: beyond the linear model
• Scalable Bayesian Variable Selection Using Nonlocal Prior Densities in Ultrahigh-Dimensional Settings
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• When do stepwise algorithms meet subset selection criteria?
• Adaptive sparse group LASSO in quantile regression
• The log-linear group-lasso estimator and its asymptotic properties
• Non-Concave Penalization in Linear Mixed-Effects Models and Regularized Selection of Fixed Effects
• Scaled Sparse Linear Regression
• A review of regularised estimation methods and cross-validation in spatiotemporal statistics
• Ultrahigh-dimensional Robust and Efficient Sparse Regression using Non-Concave Penalized Density Power Divergence
• Rodeo: Sparse Nonparametric Regression in High Dimensions
• An unbiased approach to compressed sensing
• Semiparametric Regression using Variational Approximations
• Nonuniformity of P-values Can Occur Early in Diverging Dimensions
• Rejoinder: One-step sparse estimates in nonconcave penalized likelihood models
• Sparse semiparametric regression when predictors are mixture of functional and high-dimensional variables
• Confidence Intervals for Low-Dimensional Parameters in High-Dimensional Linear Models
• Sure Independence Screening for Ultra-High Dimensional Feature Space
• Scalable Sparse Cox's Regression for Large-Scale Survival Data via Broken Adaptive Ridge
• An Overview on the Estimation of Large Covariance and Precision Matrices
• Distributional Results for Thresholding Estimators in High-Dimensional Gaussian Regression Models
• Confidence Sets Based on Thresholding Estimators in High-Dimensional Gaussian Regression Models
• Consistency of least squares estimation to the parameter for stochastic differential equations under distribution uncertainty
• A Revisit to De-biased Lasso for Generalized Linear Models
• Some Two-Step Procedures for Variable Selection in High-Dimensional Linear Regression
• Sublinear expectation linear regression
• Nearly optimal minimax estimator for high-dimensional sparse linear regression
• Partial Consistency with Sparse Incidental Parameters
• Group Variable Selection via a Hierarchical Lasso and Its Oracle Property
• Functional Group Bridge for Simultaneous Regression and Support Estimation
• Empirical likelihood based testing for regression
• Sequential Lasso for feature selection with ultra-high dimensional feature space
• False Discovery Rate Control Under General Dependence By Symmetrized Data Aggregation
• Quantile universal threshold for model selection
• Selective linear segmentation for detecting relevant parameter changes
• Direct l_(2,p)-Norm Learning for Feature Selection
• M-estimation in high-dimensional linear model
• Penalized Maximum Tangent Likelihood Estimation and Robust Variable Selection
• Variable Selection with Exponential Weights and $l_0$-Penalization
• Evaluation of Generalized Degrees of Freedom for Sparse Estimation by Replica Method
• Sparsistency of ${\ell }_1$-Regularized $M$-Estimators
• A General Theory of Concave Regularization for High Dimensional Sparse Estimation Problems
• Penalized Sparse Covariance Regression with High Dimensional Covariates
• OEM for least squares problems
• Ultra high dimensional generalized additive model: Unified Theory and Methods
• Flexible Shrinkage Estimation in High-Dimensional Varying Coefficient Models
• Multinomial Inverse Regression for Text Analysis
• Adaptive post-Dantzig estimation and prediction for non-sparse "large $p$ and small $n$" models
• A Rank-Corrected Procedure for Matrix Completion with Fixed Basis Coefficients
• The Phylogenetic LASSO and the Microbiome
• Extended BIC for linear regression models with diverging number of relevant features and high or ultra-high feature spaces
• Shrinkage Estimation and Selection for Multiple Functional Regression
• Model Selection Consistency for Cointegrating Regressions
• High Dimensional Covariance Matrix Estimation Using a Factor Model
• Variance Estimation Using Refitted Cross-validation in Ultrahigh Dimensional Regression
• Inference for possibly high-dimensional inhomogeneous Gibbs point processes
• Likelihood Adaptively Modified Penalties
• Markov Neighborhood Regression for High-Dimensional Inference
• Partial Penalized Likelihood Ratio Test under Sparse Case
• Innovated scalable efficient estimation in ultra-large Gaussian graphical models
• High-dimensional varying index coefficient quantile regression model
• Double Sparsity Kernel Learning with Automatic Variable Selection and Data Extraction
• Assessing prediction error of nonparametric regression and classification under Bregman divergence
• Variable selection in doubly truncated regression
• Upper expectation parametric regression
• Weak Signal Identification and Inference in Penalized Model Selection
• Spatial point processes intensity estimation with a diverging number of covariates
• Sparse Laplacian Shrinkage with the Graphical Lasso Estimator for Regression Problems
• Improving Network Slimming with Nonconvex Regularization
• Communication-Efficient Distributed Estimator for Generalized Linear Models with a Diverging Number of Covariates
• A Survey on Nonconvex Regularization Based Sparse and Low-Rank Recovery in Signal Processing, Statistics, and Machine Learning
• Greedy Forward Regression for Variable Screening
• Component Selection in the Additive Regression Model
• Relaxed Sparse Eigenvalue Conditions for Sparse Estimation via Non-convex Regularized Regression
• Local Linear Regression on Manifolds and its Geometric Interpretation
• Estimation and inference for high-dimensional non-sparse models
• Consistency of Bayesian Linear Model Selection With a Growing Number of Parameters
• MSP: A Multi-step Screening Procedure for Sparse Recovery
• Sufficient Dimension Reduction for Classification
• Integrated conditional moment test and beyond: when the number of covariates is divergent
• Testing (Infinitely) Many Zero Restrictions
• Variable Selection in Regression-based Estimation of Dynamic Treatment Regimes
• Shrinkage Tuning Parameter Selection in Precision Matrices Estimation
• Promoting Similarity of Sparsity Structures in Integrative Analysis with Penalization
• A Mixture of g-priors for Variable Selection when the Number of Regressors Grows with the Sample Size
• On sure early selection of the best subset
• Targeted Cross-Validation
• De-biased Lasso for Generalized Linear Models with A Diverging Number of Covariates
• Weighted residual empirical processes, martingale transformations, and model specification tests for regressions with diverging number of parameters
• A proximal-proximal majorization-minimization algorithm for nonconvex tuning-free robust regression problems
• Estimating high-dimensional Markov-switching VARs
• Sparse logistic functional principal component analysis for binary data
• Coordinate Descent for MCP/SCAD Penalized Least Squares Converges Linearly
• Inference for biased models: a quasi-instrumental variable approach
• Posterior Inference for Quantile Regression: Adaptation to Sparsity
• Generalization Bounds for High-dimensional M-estimation under Sparsity Constraint
• Selective Inference via Marginal Screening for High Dimensional Classification
• Total Variation, Adaptive Total Variation and Nonconvex Smoothly Clipped Absolute Deviation Penalty for Denoising Blocky Images
• Estimation and adaptive-to-model testing for regressions with diverging number of predictors
• Exponentially tilted likelihood inference on growing dimensional unconditional moment models
• Parameter estimation of high-dimensional linear differential equations
• Autoregressive Process Modeling via the Lasso Procedure
• Sparse and Efficient Estimation for Partial Spline Models with Increasing Dimension
• Robust Inference of Risks of Large Portfolios
• Selection by Partitioning the Solution Paths
• Semiparametric model averaging for high dimensional conditional quantile prediction