Nonlinear generalization of the monotone single index model

Abstract

Single index model is a powerful yet simple model, widely used in statistics, machine learning, and other scientific fields. It models the regression function as $g(<a,x>)$, where a is an unknown index vector and x are the features. This paper deals with a nonlinear generalization of this framework to allow for a regressor that uses multiple index vectors, adapting to local changes in the responses. To do so we exploit the conditional distribution over function-driven partitions, and use linear regression to locally estimate index vectors. We then regress by applying a kNN type estimator that uses a localized proxy of the geodesic metric. We present theoretical guarantees for estimation of local index vectors and out-of-sample prediction, and demonstrate the performance of our method with experiments on synthetic and real-world data sets, comparing it with state-of-the-art methods.

arXiv comment37 pages, 23 figures, 4 table

References in corpus (9)

• Contour regression: A general approach to dimension reduction
• Local polynomial regression on unknown manifolds
• Principal support vector machines for linear and nonlinear sufficient dimension reduction
• A general theory for nonlinear sufficient dimension reduction: Formulation and estimation
• Manifold estimation and singular deconvolution under Hausdorff loss
• Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions
• Learning gradients on manifolds
• Score estimation in the monotone single index model
• Nonparametric estimation of composite functions