Coordinate-independent sparse sufficient dimension reduction and variable selection

Abstract

Sufficient dimension reduction (SDR) in regression, which reduces the dimension by replacing original predictors with a minimal set of their linear combinations without loss of information, is very helpful when the number of predictors is large. The standard SDR methods suffer because the estimated linear combinations usually consist of all original predictors, making it difficult to interpret. In this paper, we propose a unified method - coordinate-independent sparse estimation (CISE) - that can simultaneously achieve sparse sufficient dimension reduction and screen out irrelevant and redundant variables efficiently. CISE is subspace oriented in the sense that it incorporates a coordinate-independent penalty term with a broad series of model-based and model-free SDR approaches. This results in a Grassmann manifold optimization problem and a fast algorithm is suggested. Under mild conditions, based on manifold theories and techniques, it can be shown that CISE would perform asymptotically as well as if the true irrelevant predictors were known, which is referred to as the oracle property. Simulation studies and a real-data example demonstrate the effectiveness and efficiency of the proposed approach.

Published in at http://dx.doi.org/10.1214/10-AOS826 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (5)

• Fisher Lecture: Dimension Reduction in Regression
• Contour regression: A general approach to dimension reduction
• Testing predictor contributions in sufficient dimension reduction
• Principal Fitted Components for Dimension Reduction in Regression
• Dimension reduction based on constrained canonical correlation and variable filtering

Cited by in corpus (22)

• Minimax sparse principal subspace estimation in high dimensions
• Sparse single-index model
• QUADRO: A supervised dimension reduction method via Rayleigh quotient optimization
• Penalized Orthogonal Iteration for Sparse Estimation of Generalized Eigenvalue Problem
• On the conditional distributions of low-dimensional projections from high-dimensional data
• Variable selection and estimation for semi-parametric multiple-index models
• On Stein's Identity and Near-Optimal Estimation in High-dimensional Index Models
• MM Algorithms for Distance Covariance based Sufficient Dimension Reduction and Sufficient Variable Selection
• Dimensionality determination: a thresholding double ridge ratio criterion
• Sparse Generalized Eigenvalue Problem: Optimal Statistical Rates via Truncated Rayleigh Flow
• High Dimensional Elliptical Sliced Inverse Regression in non-Gaussian Distributions
• An Inverse-free Truncated Rayleigh-Ritz Method for Sparse Generalized Eigenvalue Problem
• Dynamic Partial Sufficient Dimension Reduction
• Certifiably Optimal Sparse Sufficient Dimension Reduction
• Matrix-Variate Regressions and Envelope Models
• High-Dimensional Sparse Single-Index Regression Via Hilbert-Schmidt Independence Criterion
• Joint Dimensionality Reduction for Two Feature Vectors
• Trace Pursuit: A General Framework for Model-Free Variable Selection
• Matching Using Sufficient Dimension Reduction for Causal Inference
• On expectile-assisted inverse regression estimation for sufficient dimension reduction
• Joint Dimensionality Reduction for Separable Embedding Estimation
• Slicing-free Inverse Regression in High-dimensional Sufficient Dimension Reduction