Square Root Penalty: Adaptation to the Margin in Classification and in Edge Estimation

Abstract

We consider the problem of adaptation to the margin in binary classification. We suggest a penalized empirical risk minimization classifier that adaptively attains, up to a logarithmic factor, fast optimal rates of convergence for the excess risk, that is, rates that can be faster than n^{-1/2}, where n is the sample size. We show that our method also gives adaptive estimators for the problem of edge estimation.

Published at http://dx.doi.org/10.1214/009053604000001066 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (2)

• 2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization
• Complexity regularization via localized random penalties

Cited by in corpus (13)

• 2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization
• Estimation of high-dimensional low-rank matrices
• Fast learning rates for plug-in classifiers
• Statistical performance of support vector machines
• Optimal rates for plug-in estimators of density level sets
• Simultaneous adaptation to the margin and to complexity in classification
• Minimax fast rates for discriminant analysis with errors in variables
• Margin-adaptive model selection in statistical learning
• Risk Bounds for CART Classifiers under a Margin Condition
• Quantization via Empirical Divergence Maximization
• Minimax-Optimal Bounds for Detectors Based on Estimated Prior Probabilities
• A multivariate adaptive stochastic search method for dimensionality reduction in classification
• Discussion of ``2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization'' by V. Koltchinskii