Sparse single-index model

Abstract

Let $(\bX, Y)$ be a random pair taking values in ${\mathbb{R}}^p\times \mathbb{R}$. In the so-called single-index model, one has $Y=f^{\star}(θ^{\star T}\bX)+\bW$, where $f^{\star }$ is an unknown univariate measurable function, ${\theta }^{\star }$ is an unknown vector in ${\mathbb{R}}^d$, and $W$ denotes a random noise satisfying $\mathbb E[\bW|\bX]=0$. The single-index model is known to offer a flexible way to model a variety of high-dimensional real-world phenomena. However, despite its relative simplicity, this dimension reduction scheme is faced with severe complications as soon as the underlying dimension becomes larger than the number of observations ("$p$ larger than $n$" paradigm). To circumvent this difficulty, we consider the single-index model estimation problem from a sparsity perspective using a PAC-Bayesian approach. On the theoretical side, we offer a sharp oracle inequality, which is more powerful than the best known oracle inequalities for other common procedures of single-index recovery. The proposed method is implemented by means of the reversible jump Markov chain Monte Carlo technique and its performance is compared with that of standard procedures.

References in corpus (7)

• Simultaneous analysis of Lasso and Dantzig selector
• Coordinate-independent sparse sufficient dimension reduction and variable selection
• Sparse Regression Learning by Aggregation and Langevin Monte-Carlo
• PAC-Bayesian Bounds for Randomized Empirical Risk Minimizers
• Optimal rates and adaptation in the single-index model using aggregation
• A new algorithm for estimating the effective dimension-reduction subspace
• PAC-Bayesian Analysis of the Exploration-Exploitation Trade-off

Cited by in corpus (22)

• On the properties of variational approximations of Gibbs posteriors
• User-friendly introduction to PAC-Bayes bounds
• High-dimensional estimation with geometric constraints
• COBRA: A Combined Regression Strategy
• Bayesian Multiple Index Models for Environmental Mixtures
• Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
• Learning Single Index Models in High Dimensions
• Optimal linear estimation under unknown nonlinear transform
• Learning Sparse Nonparametric DAGs
• The generalized Lasso with non-linear observations
• Broadcasted Nonparametric Tensor Regression
• PAC-Bayesian AUC classification and scoring
• Bayesian matrix completion: prior specification
• On Oracle Property and Asymptotic Validity of Bayesian Generalized Method of Moments
• On Stein's Identity and Near-Optimal Estimation in High-dimensional Index Models
• PAC-Bayesian High Dimensional Bipartite Ranking
• High Dimensional M-Estimation with Missing Outcomes: A Semi-Parametric Framework
• De-randomized PAC-Bayes Margin Bounds: Applications to Non-convex and Non-smooth Predictors
• Misspecified Nonconvex Statistical Optimization for Phase Retrieval
• On the Exponentially Weighted Aggregate with the Laplace Prior
• Convergence guarantee for the sparse monotone single index model
• Optimal Estimation of Slope Vector in High-dimensional Linear Transformation Model