Principal support vector machines for linear and nonlinear sufficient dimension reduction

Abstract

We introduce a principal support vector machine (PSVM) approach that can be used for both linear and nonlinear sufficient dimension reduction. The basic idea is to divide the response variables into slices and use a modified form of support vector machine to find the optimal hyperplanes that separate them. These optimal hyperplanes are then aligned by the principal components of their normal vectors. It is proved that the aligned normal vectors provide an unbiased, $\sqrt{n}$-consistent, and asymptotically normal estimator of the sufficient dimension reduction space. The method is then generalized to nonlinear sufficient dimension reduction using the reproducing kernel Hilbert space. In that context, the aligned normal vectors become functions and it is proved that they are unbiased in the sense that they are functions of the true nonlinear sufficient predictors. We compare PSVM with other sufficient dimension reduction methods by simulation and in real data analysis, and through both comparisons firmly establish its practical advantages.

Published in at http://dx.doi.org/10.1214/11-AOS932 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (6)

• Kernel dimension reduction in regression
• Fisher Lecture: Dimension Reduction in Regression
• Contour regression: A general approach to dimension reduction
• Principal Fitted Components for Dimension Reduction in Regression
• Dimension reduction for nonelliptically distributed predictors
• An RKHS formulation of the inverse regression dimension-reduction problem

Cited by in corpus (9)

• A general theory for nonlinear sufficient dimension reduction: Formulation and estimation
• On efficient dimension reduction with respect to a statistical functional of interest
• On the conditional distributions of low-dimensional projections from high-dimensional data
• Nonlinear generalization of the monotone single index model
• Learning Integral Representations of Gaussian Processes
• Approximate Bayesian Computation via Sufficient Dimension Reduction
• Sufficient Dimension Reduction for High-Dimensional Regression and Low-Dimensional Embedding: Tutorial and Survey
• On sufficient dimension reduction via principal asymmetric least squares
• Predictive Directions for Individualized Treatment Selection in Clinical Trials