On Stein's Identity and Near-Optimal Estimation in High-dimensional Index Models

Abstract

We consider estimating the parametric components of semi-parametric multiple index models in a high-dimensional and non-Gaussian setting. Such models form a rich class of non-linear models with applications to signal processing, machine learning and statistics. Our estimators leverage the score function based first and second-order Stein's identities and do not require the covariates to satisfy Gaussian or elliptical symmetry assumptions common in the literature. Moreover, to handle score functions and responses that are heavy-tailed, our estimators are constructed via carefully thresholding their empirical counterparts. We show that our estimator achieves near-optimal statistical rate of convergence in several settings. We supplement our theoretical results via simulation experiments that confirm the theory.

References in corpus (13)

• Square Deal: Lower Bounds and Improved Relaxations for Tensor Recovery
• Coordinate-independent sparse sufficient dimension reduction and variable selection
• Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods
• A Partial Derandomization of PhaseLift using Spherical Designs
• The LASSO with Non-linear Measurements is Equivalent to One With Linear Measurements
• The lower tail of random quadratic forms, with applications to ordinary least squares and restricted eigenvalue properties
• Sparse single-index model
• A Shrinkage Principle for Heavy-Tailed Data: High-Dimensional Robust Low-Rank Matrix Recovery
• Non-Convex Projected Gradient Descent for Generalized Low-Rank Tensor Regression
• On consistency and sparsity for sliced inverse regression in high dimensions
• Structured signal recovery from non-linear and heavy-tailed measurements
• On the optimality of sliced inverse regression in high dimensions
• Optimal Rates of Convergence for Noisy Sparse Phase Retrieval via Thresholded Wirtinger Flow

Cited by in corpus (8)

• Generic Error Bounds for the Generalized Lasso with Sub-Exponential Data
• Structured Recovery with Heavy-tailed Measurements: A Thresholding Procedure and Optimal Rates
• The Generalized Lasso with Nonlinear Observations and Generative Priors
• Distributed Estimation for Principal Component Analysis: an Enlarged Eigenspace Analysis
• The Mismatch Principle: The Generalized Lasso Under Large Model Uncertainties
• Learning Polynomials of Few Relevant Dimensions
• Quickly Finding the Best Linear Model in High Dimensions
• II. High Dimensional Estimation under Weak Moment Assumptions: Structured Recovery and Matrix Estimation