The Dantzig selector and sparsity oracle inequalities

Abstract

Let \[Y_j=f_*(X_j)+ξ_j,\qquad j=1,...,n,\] where $X,X_1,...,X_n$ are i.i.d. random variables in a measurable space $(S,\mathcal{A})$ with distribution $\Pi$ and $\xi ,{\xi }_1,...,{\xi }_n$ are i.i.d. random variables with $\mathbb{E}\xi =0$ independent of $(X_1,...,X_n).$ Given a dictionary $h_1,...,h_N:S\mapsto \mathbb{R},$ let $f_{\lambda }:={\sum }_{j=1}^N{\lambda }_j{h}_j$, $\lambda =({\lambda }_1,...,{\lambda }_N)\in {\mathbb{R}}^N.$ Given $\epsilon >0,$ define \[\hatΛ_{\varepsilon}:=\Biggl\{\lam bda\in{\mathbb{R}}^N:\max_{1\leq k\leq N}\Biggl|n^{-1}\sum_{j=1}^n\big l(f_λ(X_j)-Y_j\bigr)h_k(X_j)\Biggr|\leq\varepsilon \Biggr\}\] and \[\hatλ:=\hatλ^{\varepsilon}\in \operatorname {Arg min}\limits_{λ\in\hatΛ_{\varepsilon}}\|λ\|_{\ell_1}.\] In the case where $f_{\ast }:=f_{{\lambda }^{\ast }},{\lambda }^{\ast }\in {\mathbb{R}}^N,$ Candes and Tao [Ann. Statist. 35 (2007) 2313--2351] suggested using $\hat{\lambda }$ as an estimator of ${\lambda }^{\ast }.$ They called this estimator ``the Dantzig selector''. We study the properties of $f_{\hat{\lambda }}$ as an estimator of $f_{\ast }$ for regression models with random design, extending some of the results of Candes and Tao (and providing alternative proofs of these results).

Published in at http://dx.doi.org/10.3150/09-BEJ187 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)

References in corpus (2)

• High-dimensional generalized linear models and the lasso
• Sparsity oracle inequalities for the Lasso

Cited by in corpus (5)

• Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
• Some sharp performance bounds for least squares regression with $L_1$ regularization
• Adaptive Lasso for High Dimensional Regression and Gaussian Graphical Modeling
• Sparse recovery in convex hulls via entropy penalization
• High-dimensional stochastic optimization with the generalized Dantzig estimator