Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
arXiv:1511.04514
Abstract
We study parameter estimation and asymptotic inference for sparse nonlinear regression. More specifically, we assume the data are given by , where is nonlinear. To recover , we propose an -regularized least-squares estimator. Unlike classical linear regression, the corresponding optimization problem is nonconvex because of the nonlinearity of . In spite of the nonconvexity, we prove that under mild conditions, every stationary point of the objective enjoys an optimal statistical rate of convergence. In addition, we provide an efficient algorithm that provably converges to a stationary point. We also access the uncertainty of the obtained estimator. Specifically, based on any stationary point of the objective, we construct valid hypothesis tests and confidence intervals for the low dimensional components of the high-dimensional parameter . Detailed numerical results are provided to back up our theory.
arXiv comment32 pages, 2 figures, 1 table
References in corpus (16)
- A significance test for the lasso
- GESPAR: Efficient Phase Retrieval of Sparse Signals
- Low-rank Solutions of Linear Matrix Equations via Procrustes Flow
- Compressive Phase Retrieval From Squared Output Measurements Via Semidefinite Programming
- Sparse Phase Retrieval from Short-Time Fourier Measurements
- Exponential Screening and optimal rates of sparse estimation
- Efficient Learning of Generalized Linear and Single Index Models with Isotonic Regression
- High-dimensional estimation with geometric constraints
- Exact Post-Selection Inference for Sequential Regression Procedures
- Sparse single-index model
- Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity
- A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models
- The generalized Lasso with non-linear observations
- Optimal Rates of Convergence for Noisy Sparse Phase Retrieval via Thresholded Wirtinger Flow
- A Likelihood Ratio Framework for High Dimensional Semiparametric Regression
- Sparse Recovery with Linear and Nonlinear Observations: Dependent and Noisy Data
Cited by in corpus (11)
- Global and Simultaneous Hypothesis Testing for High-Dimensional Logistic Regression Models
- The Landscape of Empirical Risk for Non-convex Losses
- Successive Convex Approximation Algorithms for Sparse Signal Estimation with Nonconvex Regularizations
- Fast Algorithms for Demixing Sparse Signals from Nonlinear Observations
- Rational Optimization for Nonlinear Reconstruction with Approximate Penalization
- Generic Error Bounds for the Generalized Lasso with Sub-Exponential Data
- Fast Low-Rank Matrix Estimation without the Condition Number
- The Mismatch Principle: The Generalized Lasso Under Large Model Uncertainties
- Iterative Thresholding for Demixing Structured Superpositions in High Dimensions
- Misspecified Nonconvex Statistical Optimization for Phase Retrieval
- Semiparametric Wavelet-based JPEG IV Estimator for endogenously truncated data