Bandwidth choice for nonparametric classification

Abstract

It is shown that, for kernel-based classification with univariate distributions and two populations, optimal bandwidth choice has a dichotomous character. If the two densities cross at just one point, where their curvatures have the same signs, then minimum Bayes risk is achieved using bandwidths which are an order of magnitude larger than those which minimize pointwise estimation error. On the other hand, if the curvature signs are different, or if there are multiple crossing points, then bandwidths of conventional size are generally appropriate. The range of different modes of behavior is narrower in multivariate settings. There, the optimal size of bandwidth is generally the same as that which is appropriate for pointwise density estimation. These properties motivate empirical rules for bandwidth choice.

Published at http://dx.doi.org/10.1214/009053604000000959 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Cited by in corpus (10)

• Choice of neighbor order in nearest-neighbor classification
• Optimal weighted nearest neighbour classifiers
• Asymptotic normality of plug-in level set estimates
• Unexpected properties of bandwidth choice when smoothing discrete data for constructing a functional data classifier
• Optimal properties of centroid-based classifiers for very high-dimensional data
• Nearest-Neighbor Sample Compression: Efficiency, Consistency, Infinite Dimensions
• Asymptotics and Optimal Bandwidth for Nonparametric Estimation of Density Level Sets
• Nonparametric Estimation of Surface Integrals on Level Sets
• Fast Multi-Class Probabilistic Classifier by Sparse Non-parametric Density Estimation
• Nonparametric Confidence Regions for Level Sets: Statistical Properties and Geometry