Multivariate convex regression: global risk bounds and adaptation

Abstract

We study the problem of estimating a multivariate convex function defined on a convex body in a regression setting with random design. We are interested in optimal rates of convergence under a squared global continuous $l_2$ loss in the multivariate setting $(d\ge 2)$. One crucial fact is that the minimax risks depend heavily on the shape of the support of the regression function. It is shown that the global minimax risk is on the order of $n^{-2/(d+1)}$ when the support is sufficiently smooth, but that the rate $n^{-4/(d+4)}$ is when the support is a polytope. Such differences in rates are due to difficulties in estimating the regression function near the boundary of smooth regions. We then study the natural bounded least squares estimators (BLSE): we show that the BLSE nearly attains the optimal rates of convergence in low dimensions, while suffering rate-inefficiency in high dimensions. We show that the BLSE adapts nearly parametrically to polyhedral functions when the support is polyhedral in low dimensions by a local entropy method. We also show that the boundedness constraint cannot be dropped when risk is assessed via continuous $l_2$ loss. Given rate sub-optimality of the BLSE in higher dimensions, we further study rate-efficient adaptive estimation procedures. Two general model selection methods are developed to provide sieved adaptive estimators (SAE) that achieve nearly optimal rates of convergence for particular "regular" classes of convex functions, while maintaining nearly parametric rate-adaptivity to polyhedral functions in arbitrary dimensions. Interestingly, the uniform boundedness constraint is unnecessary when risks are measured in discrete $l_2$ norms.

75 pages

References in corpus (5)

• Approximation and Estimation of s-Concave Densities via Rényi Divergences
• Faithful Variable Screening for High-Dimensional Convex Regression
• Entropy of convex functions on $R^d$
• Adaptive estimation of planar convex sets
• Covering numbers of $L_p$-balls of convex sets and functions

Cited by in corpus (15)

• Optimality of Maximum Likelihood for Log-Concave Density Estimation and Bounded Convex Regression
• Max-Affine Regression: Provable, Tractable, and Near-Optimal Statistical Estimation
• On Estimation of Isotonic Piecewise Constant Signals
• A Projection Framework for Testing Shape Restrictions That Form Convex Cones
• Multivariate Distributionally Robust Convex Regression under Absolute Error Loss
• Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators
• Handling Hard Affine SDP Shape Constraints in RKHSs
• Confidence intervals for multiple isotonic regression and other monotone models
• Oracle posterior contraction rates under hierarchical priors
• Nonparametric, tuning-free estimation of S-shaped functions
• Hard Shape-Constrained Kernel Machines
• Stratified incomplete local simplex tests for curvature of nonparametric multiple regression
• Estimating Piecewise Monotone Signals
• Least Squares Estimation of a Quasiconvex Regression Function
• On the Minimal Error of Empirical Risk Minimization