Robustness of shape-restricted regression estimators: an envelope perspective

Abstract

Classical least squares estimators are well-known to be robust with respect to moment assumptions concerning the error distribution in a wide variety of finite-dimensional statistical problems; generally only a second moment assumption is required for least squares estimators to maintain the same rate of convergence that they would satisfy if the errors were assumed to be Gaussian. In this paper, we give a geometric characterization of the robustness of shape-restricted least squares estimators (LSEs) to error distributions with an $L_{2,1}$ moment, in terms of the `localized envelopes' of the model. This envelope perspective gives a systematic approach to proving oracle inequalities for the LSEs in shape-restricted regression problems in the random design setting, under a minimal $L_{2,1}$ moment assumption on the errors. The canonical isotonic and convex regression models, and a more challenging additive regression model with shape constraints are studied in detail. Strikingly enough, in the additive model both the adaptation and robustness properties of the LSE can be preserved, up to error distributions with an $L_{2,1}$ moment, for estimating the shape-constrained proxy of the marginal $L_2$ projection of the true regression function. This holds essentially regardless of whether or not the additive model structure is correctly specified. The new envelope perspective goes beyond shape constrained models. Indeed, at a general level, the localized envelopes give a sharp characterization of the convergence rate of the $L_2$ loss of the LSE between the worst-case rate as suggested by the recent work of the authors [25], and the best possible parametric rate.

arXiv comment44 pages, 1 figure

References in corpus (8)

• 2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization
• Risk bounds for statistical learning
• On risk bounds in isotonic and other shape restricted regression problems
• Concentration inequalities and asymptotic results for ratio type empirical processes
• A new perspective on least squares under convex constraint
• Nonparametric Shape-restricted Regression
• On Estimation of Isotonic Piecewise Constant Signals
• Convergence rates of least squares regression estimators with heavy-tailed errors

Cited by in corpus (3)

• Convergence rates of least squares regression estimators with heavy-tailed errors
• Semiparametric Efficiency in Convexity Constrained Single Index Model
• Multiplier U-processes: sharp bounds and applications