The Dantzig selector and sparsity oracle inequalities

Abstract

Let \[Y_j=f_*(X_j)+ξ_j,\qquad j=1,...,n,\] where $X,X_1,...,X_n$ are i.i.d. random variables in a measurable space $(S,\mathcal{A})$ with distribution $\Pi$ and $\xi ,{\xi }_1,...,{\xi }_n$ are i.i.d. random variables with $\mathbb{E}\xi =0$ independent of $(X_1,...,X_n).$ Given a dictionary $h_1,...,h_N:S\mapsto \mathbb{R},$ let $f_{\lambda }:={\sum }_{j=1}^N{\lambda }_j{h}_j$, $\lambda =({\lambda }_1,...,{\lambda }_N)\in {\mathbb{R}}^N.$ Given $\epsilon >0,$ define \[\hatΛ_{\varepsilon}:=\Biggl\{\lam bda\in{\mathbb{R}}^N:\max_{1\leq k\leq N}\Biggl|n^{-1}\sum_{j=1}^n\big l(f_λ(X_j)-Y_j\bigr)h_k(X_j)\Biggr|\leq\varepsilon \Biggr\}\] and \[\hatλ:=\hatλ^{\varepsilon}\in \operatorname {Arg min}\limits_{λ\in\hatΛ_{\varepsilon}}\|λ\|_{\ell_1}.\] In the case where $f_{\ast }:=f_{{\lambda }^{\ast }},{\lambda }^{\ast }\in {\mathbb{R}}^N,$ Candes and Tao [Ann. Statist. 35 (2007) 2313--2351] suggested using $\hat{\lambda }$ as an estimator of ${\lambda }^{\ast }.$ They called this estimator ``the Dantzig selector''. We study the properties of $f_{\hat{\lambda }}$ as an estimator of $f_{\ast }$ for regression models with random design, extending some of the results of Candes and Tao (and providing alternative proofs of these results).

Published in at http://dx.doi.org/10.3150/09-BEJ187 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)

References in corpus (3)

• Simultaneous analysis of Lasso and Dantzig selector
• High-dimensional generalized linear models and the lasso
• Sparsity oracle inequalities for the Lasso

Cited by in corpus (52)

• Simultaneous analysis of Lasso and Dantzig selector
• On the conditions used to prove oracle results for the Lasso
• Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors
• A Selective Review of Group Selection in High-Dimensional Models
• Asymptotic normality and optimalities in estimation of large Gaussian graphical models
• Statistical significance in high-dimensional linear models
• Sparsity in multiple kernel learning
• L1-Penalization for Mixture Regression Models
• Sparse recovery under matrix uncertainty
• Regularization for Cox's proportional hazards model with NP-dimensionality
• High-dimensional instrumental variables regression and confidence sets
• Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
• Some sharp performance bounds for least squares regression with $L_1$ regularization
• Oracle inequalities for the lasso in the Cox model
• A new perspective on least squares under convex constraint
• A framework to characterize performance of LASSO algorithms
• Exponential Screening and optimal rates of sparse estimation
• Sparse Matrix Inversion with Scaled Lasso
• Adaptive Lasso for High Dimensional Regression and Gaussian Graphical Modeling
• Factor models and variable selection in high-dimensional regression analysis
• Goodness of fit tests for high-dimensional linear models
• Thresholded Lasso for high dimensional variable selection and statistical estimation
• Mirror averaging with sparsity priors
• Assumptionless consistency of the Lasso
• Pac-bayesian bounds for sparse regression estimation with exponential weights
• Sparse recovery in convex hulls via entropy penalization
• Accuracy guarantees for L1-recovery
• Quasi-Likelihood and/or Robust Estimation in High Dimensions
• Estimation And Selection Via Absolute Penalized Convex Minimization And Its Multistage Adaptive Applications
• Confidence Intervals for Low-Dimensional Parameters in High-Dimensional Linear Models
• The adaptive and the thresholded Lasso for potentially misspecified models
• General nonexact oracle inequalities for classes with a subexponential envelope
• Adaptive Minimax Estimation over Sparse ${\ell }_q$-Hulls
• A performance analysis framework for SOCP algorithms in noisy compressed sensing
• Sample average approximation with heavier tails II: localization in stochastic convex optimization and persistence results for the Lasso
• Link Prediction in Graphs with Autoregressive Features
• Regularity Properties for Sparse Regression
• A General Theory of Concave Regularization for High Dimensional Sparse Estimation Problems
• High-dimensional stochastic optimization with the generalized Dantzig estimator
• Fast-rate and optimistic-rate error bounds for L1-regularized regression
• The Benefit of Group Sparsity in Group Inference with De-biased Scaled Group Lasso
• High-dimensional additive hazard models and the Lasso
• Estimation of low rank density matrices by Pauli measurements
• High-dimensional Adaptive Minimax Sparse Estimation with Interactions
• Relaxed Sparse Eigenvalue Conditions for Sparse Estimation via Non-convex Regularized Regression
• Censored linear model in high dimensions
• High-Dimensional Semiparametric Selection Models: Estimation Theory with an Application to the Retail Gasoline Market
• The $l_q$ consistency of the Dantzig Selector for Cox's Proportional Hazards Model
• Adaptive Estimation In High-Dimensional Additive Models With Multi-Resolution Group Lasso
• A Proximal-Gradient Homotopy Method for the Sparse Least-Squares Problem
• $L_1$-Penalization in Functional Linear Regression with Subgaussian Design
• A problem dependent analysis of SOCP algorithms in noisy compressed sensing