Bandwidth selection in kernel empirical risk minimization via the gradient
arXiv:1401.6882 · doi:10.1214/15-AOS1318
Abstract
In this paper, we deal with the data-driven selection of multidimensional and possibly anisotropic bandwidths in the general framework of kernel empirical risk minimization. We propose a universal selection rule, which leads to optimal adaptive results in a large variety of statistical models such as nonparametric robust regression and statistical learning with errors in variables. These results are stated in the context of smooth loss functions, where the gradient of the risk appears as a good criterion to measure the performance of our estimators. The selection rule consists of a comparison of gradient empirical risks. It can be viewed as a nontrivial improvement of the so-called Goldenshluger-Lepski method to nonlinear estimators. Furthermore, one main advantage of our selection rule is the nondependency on the Hessian matrix of the risk, usually involved in standard adaptive procedures.
arXiv commentPublished at http://dx.doi.org/10.1214/15-AOS1318 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)
References in corpus (12)
- 2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization
- Risk bounds for statistical learning
- Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality
- Statistical performance of support vector machines
- Data-driven calibration of penalties for least-squares regression
- Universal pointwise selection rule in multivariate function estimation
- Estimation of distributions, moments and quantiles in deconvolution problems
- Simultaneous adaptation to the margin and to complexity in classification
- Adaptive quantile estimation in deconvolution with unknown error distribution
- Uniform bounds for norms of sums of independent random functions
- Minimax fast rates for discriminant analysis with errors in variables
- A robust, adaptive M-estimator for pointwise estimation in heteroscedastic regression