Learning Single Index Models in High Dimensions
arXiv:1506.08910
Abstract
Single Index Models (SIMs) are simple yet flexible semi-parametric models for classification and regression. Response variables are modeled as a nonlinear, monotonic function of a linear combination of features. Estimation in this context requires learning both the feature weights, and the nonlinear function. While methods have been described to learn SIMs in the low dimensional regime, a method that can efficiently learn SIMs in high dimensions has not been forthcoming. We propose three variants of a computationally and statistically efficient algorithm for SIM inference in high dimensions. We establish excess risk bounds for the proposed algorithms and experimentally validate the advantages that our SIM learning methods provide relative to Generalized Linear Model (GLM) and low dimensional SIM based learning methods.
16 pages, 2 figures, 1 table
References in corpus (5)
- A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
- Efficient Learning of Generalized Linear and Single Index Models with Isotonic Regression
- High-dimensional estimation with geometric constraints
- Sparse single-index model
- Classification with Sparse Overlapping Groups
Cited by in corpus (10)
- Signal Processing on Graphs: Causal Modeling of Unstructured Data
- Fast Algorithms for Demixing Sparse Signals from Nonlinear Observations
- SILVar: Single Index Latent Variable Models
- Least squares estimation in the monotone single index model
- Inference In High-dimensional Single-Index Models Under Symmetric Designs
- Semiparametric Efficiency in Convexity Constrained Single Index Model
- On Learning High Dimensional Structured Single Index Models
- A Robust Time Series Model with Outliers and Missing Entries
- Single Index Latent Variable Models for Network Topology Inference
- Quickly Finding the Best Linear Model in High Dimensions