On risk bounds in isotonic and other shape restricted regression problems

Abstract

We consider the problem of estimating an unknown $\theta \in {\mathbb{R}}^n$ from noisy observations under the constraint that $\theta$ belongs to certain convex polyhedral cones in ${\mathbb{R}}^n$. Under this setting, we prove bounds for the risk of the least squares estimator (LSE). The obtained risk bound behaves differently depending on the true sequence $\theta$ which highlights the adaptive behavior of $\theta$. As special cases of our general result, we derive risk bounds for the LSE in univariate isotonic and convex regression. We study the risk bound in isotonic regression in greater detail: we show that the isotonic LSE converges at a whole range of rates from $\log n/n$ (when $\theta$ is constant) to $n^{-2/3}$ (when $\theta$ is uniformly increasing in a certain sense). We argue that the bound presents a benchmark for the risk of any estimator in isotonic regression by proving nonasymptotic local minimax lower bounds. We prove an analogue of our bound for model misspecification where the true $\theta$ is not necessarily nondecreasing.

Published at http://dx.doi.org/10.1214/15-AOS1324 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (1)

• Various thresholds for ${\ell }_1$-optimization in compressed sensing

Cited by in corpus (37)

• On the Prediction Performance of the Lasso
• A new perspective on least squares under convex constraint
• Multivariate convex regression: global risk bounds and adaptation
• Unbiased Markov chain Monte Carlo with couplings
• Minimax Estimation of Conditional Moment Models
• On matrix estimation under monotonicity constraints
• A review on minimax rates in change point detection and localisation
• Optimal Permutation Recovery in Permuted Monotone Matrix Model
• Sharp oracle bounds for monotone and convex regression through aggregation
• Nonparametric Shape-restricted Regression
• Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy-Krause variation
• On Estimation of Isotonic Piecewise Constant Signals
• Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators
• Bayesian fractional posteriors
• Isotonic Regression in Multi-Dimensional Spaces and Graphs
• Contraction and uniform convergence of isotonic regression
• Robustness of shape-restricted regression estimators: an envelope perspective
• General framework for projection structures
• Confidence intervals for multiple isotonic regression and other monotone models
• Rates of convergence of rho-estimators for sets of densities satisfying shape constraints
• Element-wise estimation error of a total variation regularized estimator for change point detection
• Hard Shape-Constrained Kernel Machines
• Nonparametric, tuning-free estimation of S-shaped functions
• Regret Minimization in Isotonic, Heavy-Tailed Contextual Bandits via Adaptive Confidence Bands
• Oracle posterior contraction rates under hierarchical priors
• On a phase transition in general order spline regression
• Adaptive estimation of planar convex sets
• The bias of isotonic regression
• Efficient Algorithms for Multidimensional Segmented Regression
• Improved Prediction and Network Estimation Using the Monotone Single Index Multi-variate Autoregressive Model
• Model Repair: Robust Recovery of Over-Parameterized Statistical Models
• Convergence guarantee for the sparse monotone single index model
• Predicting Online Item-choice Behavior: A Shape-restricted Regression Perspective
• Estimating Piecewise Monotone Signals
• Testing against a linear regression model using ideas from shape-restricted estimation
• Least Squares Estimation of a Quasiconvex Regression Function
• Piecewise monotone estimation in one-parameter exponential family