Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators

Abstract

Under the usual nonparametric regression model with Gaussian errors, Least Squares Estimators (LSEs) over natural subclasses of convex functions are shown to be suboptimal for estimating a $d$-dimensional convex function in squared error loss when the dimension $d$ is 5 or larger. The specific function classes considered include: (i) bounded convex functions supported on a polytope (in random design), (ii) Lipschitz convex functions supported on any convex domain (in random design), (iii) convex functions supported on a polytope (in fixed design). For each of these classes, the risk of the LSE is proved to be of the order $n^{-2/d}$ (up to logarithmic factors) while the minimax risk is $n^{-4/(d+4)}$, when $d\ge 5$. In addition, the first rate of convergence results (worst case and adaptive) for the unrestricted convex LSE are established in fixed-design for polytopal domains for all $d\ge 1$. Some new metric entropy results for convex functions are also proved which are of independent interest.

To appear in the Annals of Statistics

References in corpus (6)

• On risk bounds in isotonic and other shape restricted regression problems
• A new perspective on least squares under convex constraint
• Multivariate convex regression: global risk bounds and adaptation
• Optimality of Maximum Likelihood for Log-Concave Density Estimation and Bounded Convex Regression
• Nonparametric Least Squares Estimation of a Multivariate Convex Regression Function
• Entropy of convex functions on $R^d$

Cited by in corpus (5)

• Rates of Estimation of Optimal Transport Maps using Plug-in Estimators via Barycentric Projections
• Minimax Rates for Conditional Density Estimation via Empirical Entropy
• On Suboptimality of Least Squares with Application to Estimation of Convex Bodies
• Nonparametric, tuning-free estimation of S-shaped functions
• On the Minimal Error of Empirical Risk Minimization