Composite quantile regression and the oracle Model Selection Theory

Abstract

Coefficient estimation and variable selection in multiple linear regression is routinely done in the (penalized) least squares (LS) framework. The concept of model selection oracle introduced by Fan and Li [J. Amer. Statist. Assoc. 96 (2001) 1348--1360] characterizes the optimal behavior of a model selection procedure. However, the least-squares oracle theory breaks down if the error variance is infinite. In the current paper we propose a new regression method called composite quantile regression (CQR). We show that the oracle model selection theory using the CQR oracle works beautifully even when the error variance is infinite. We develop a new oracular procedure to achieve the optimal properties of the CQR oracle. When the error variance is finite, CQR still enjoys great advantages in terms of estimation efficiency. We show that the relative efficiency of CQR compared to the least squares is greater than 70% regardless the error distribution. Moreover, CQR could be much more efficient and sometimes arbitrarily more efficient than the least squares. The same conclusions hold when comparing a CQR-oracular estimator with a LS-oracular estimator.

Published in at http://dx.doi.org/10.1214/07-AOS507 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (2)

• Composite quantile regression and the oracle Model Selection Theory
• Statistical Challenges with High Dimensionality: Feature Selection in Knowledge Discovery

Cited by in corpus (45)

• Composite quantile regression and the oracle Model Selection Theory
• Strong oracle optimality of folded concave penalized estimation
• Robust rank correlation based screening
• New efficient estimation and variable selection methods for semiparametric varying-coefficient partially linear models
• Penalized Composite Quasi-Likelihood for Ultrahigh-Dimensional Variable Selection
• Adaptive robust variable selection
• Loss minimization and parameter estimation with heavy tails
• Bayesian empirical likelihood for quantile regression
• A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)
• An adaptive composite quantile approach to dimension reduction
• Distributed High-dimensional Regression Under a Quantile Loss Function
• Confidence Sets Based on Sparse Estimators Are Necessarily Large
• Large Covariance Estimation through Elliptical Factor Models
• Robust Variable Selection and Estimation Via Adaptive Elastic Net S-Estimators for Linear Regression
• On efficient dimension reduction with respect to a statistical functional of interest
• Inference in Functional Linear Quantile Regression
• A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression
• Adaptive LASSO model selection in a multiphase quantile regression
• Robust Estimation of High-Dimensional Mean Regression
• Variance Reduced Median-of-Means Estimator for Byzantine-Robust Distributed Inference
• A New Family of Error Distributions for Bayesian Quantile Regression
• High-Dimensional Composite Quantile Regression: Optimal Statistical Guarantees and Fast Algorithms
• Quantile autoregressive conditional heteroscedasticity
• High-Dimensional Quantile Regression: Convolution Smoothing and Concave Regularization
• Adaptive elastic-net selection in a quantile model with diverging number of variable groups
• Penalized Maximum Tangent Likelihood Estimation and Robust Variable Selection
• Renewable Composite Quantile Method and Algorithm for Nonparametric Models with Streaming Data
• Quantile Regression for General Spatial Panel Data Models with Fixed Effects
• Jackknife Model Averaging for Composite Quantile Regression
• Wild Residual Bootstrap Inference for Penalized Quantile Regression with Heteroscedastic Errors
• Adaptive minimax optimality in statistical inverse problems via SOLIT -- Sharp Optimal Lepskii-Inspired Tuning
• On the Distribution of the Adaptive LASSO Estimator
• Sparse Wavelet Estimation in Quantile Regression with Multiple Functional Predictors
• Model selection in high-dimensional quantile regression with seamless $L_0$ penalty
• Marked empirical processes for non-stationary time series
• Robust Variable Selection Criteria for the Penalized Regression
• Wilcoxon-type Multivariate Cluster Elastic Net
• High-dimensional Adaptive Minimax Sparse Estimation with Interactions
• Interquantile Shrinkage in Spatial Quantile Autoregressive Regression models
• A Global Bias-Correction DC Method for Biased Estimation under Memory Constraint
• Quantile index regression
• Quantile regression with generated dependent variable and covariates
• Asymptotic Composite Estimation
• Penalized regression with multiple loss functions and selection by vote
• Nearly Semiparametric Efficient Estimation of Quantile Regression