Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference

Abstract

We study parameter estimation and asymptotic inference for sparse nonlinear regression. More specifically, we assume the data are given by $y=f(x^{\top }{\beta }^{\ast })+\epsilon$, where $f$ is nonlinear. To recover ${\beta }^{\ast }$, we propose an ${\ell }_1$-regularized least-squares estimator. Unlike classical linear regression, the corresponding optimization problem is nonconvex because of the nonlinearity of $f$. In spite of the nonconvexity, we prove that under mild conditions, every stationary point of the objective enjoys an optimal statistical rate of convergence. In addition, we provide an efficient algorithm that provably converges to a stationary point. We also access the uncertainty of the obtained estimator. Specifically, based on any stationary point of the objective, we construct valid hypothesis tests and confidence intervals for the low dimensional components of the high-dimensional parameter ${\beta }^{\ast }$. Detailed numerical results are provided to back up our theory.

32 pages, 2 figures, 1 table

References in corpus (7)

• Low-rank Solutions of Linear Matrix Equations via Procrustes Flow
• Sparse Phase Retrieval from Short-Time Fourier Measurements
• Efficient Learning of Generalized Linear and Single Index Models with Isotonic Regression
• A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models
• The generalized Lasso with non-linear observations
• Optimal Rates of Convergence for Noisy Sparse Phase Retrieval via Thresholded Wirtinger Flow
• Sparse Recovery with Linear and Nonlinear Observations: Dependent and Noisy Data

Cited by in corpus (11)

• Global and Simultaneous Hypothesis Testing for High-Dimensional Logistic Regression Models
• The Landscape of Empirical Risk for Non-convex Losses
• Successive Convex Approximation Algorithms for Sparse Signal Estimation with Nonconvex Regularizations
• Fast Algorithms for Demixing Sparse Signals from Nonlinear Observations
• Rational Optimization for Nonlinear Reconstruction with Approximate ${\ell }_0$ Penalization
• Generic Error Bounds for the Generalized Lasso with Sub-Exponential Data
• Fast Low-Rank Matrix Estimation without the Condition Number
• The Mismatch Principle: The Generalized Lasso Under Large Model Uncertainties
• Iterative Thresholding for Demixing Structured Superpositions in High Dimensions
• Misspecified Nonconvex Statistical Optimization for Phase Retrieval
• Semiparametric Wavelet-based JPEG IV Estimator for endogenously truncated data