Tight conditions for consistent variable selection in high dimensional nonparametric regression

Abstract

We address the issue of variable selection in the regression model with very high ambient dimension, i.e., when the number of covariates is very large. The main focus is on the situation where the number of relevant covariates, called intrinsic dimension, is much smaller than the ambient dimension. Without assuming any parametric form of the underlying regression function, we get tight conditions making it possible to consistently estimate the set of relevant variables. These conditions relate the intrinsic dimension to the ambient dimension and to the sample size. The procedure that is provably consistent under these tight conditions is simple and is based on comparing the empirical Fourier coefficients with an appropriately chosen threshold value.

References in corpus (9)

• Nearly unbiased variable selection under minimax concave penalty
• The composite absolute penalties family for grouped and hierarchical variable selection
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• Structured Variable Selection with Sparsity-Inducing Norms
• Support union recovery in high-dimensional multivariate regression
• High-Dimensional Non-Linear Variable Selection through Hierarchical Kernel Learning
• Selection of variables and dimension reduction in high-dimensional non-parametric regression
• Iterative Feature Selection In Least Square Regression Estimation
• Oracle Inequalities and Optimal Inference under Group Sparsity

Cited by in corpus (5)

• Tight conditions for consistency of variable selection in the context of high dimensionality
• Subspace Embeddings and ${\ell }_p$-Regression Using Exponential Random Variables
• Learning Sparse Additive Models with Interactions in High Dimensions
• A polynomial-time algorithm for learning nonparametric causal graphs
• Learning general sparse additive models from point queries in high dimensions