Conditional regression for the Nonlinear Single-Variable Model

Abstract

Regressing a function $F$ on ${\mathbb{R}}^d$ without incurring the statistical and computational curse of dimensionality requires exploitable structure. Compositional models $F=f\circ g$ in which $g$ has a low-dimensional range include classical single- and multi-index models as well as certain neural networks; while the case of linear $g$ is well understood, substantially less is known for nonlinear $g$. We study the model $F(X)=f({\Pi }_{\gamma }X)$, where ${\Pi }_{\gamma }$ is the closest-point coordinate associated with an unknown regular curve $\gamma$, and $f$ is an unknown one-dimensional link function. The predictor $X$ need not be intrinsically low-dimensional and may have full-dimensional variation throughout a tubular neighborhood of the curve. We construct a nonparametric estimator based on response slicing, local principal component analysis, data-adaptive slice assignment, and one-dimensional local polynomial regression. Under coarse monotonicity of $f$ and sufficient variation normal to the curve relative to the observational noise and the coarse-monotonicity scale, the estimator attains, up to logarithmic factors, the minimax-optimal one-dimensional mean squared rate down to an explicit geometry- and noise-dependent saturation level. When the normal-variation condition is removed, we prove a complementary guarantee for the wide-slice regime. The estimator can be constructed in time $\mathcal{O}(d^{2}n\log n)$, and the constants and sample-size thresholds in our bounds depend at most polynomially on the ambient dimension $d$.

75 pages, 13 figures