Rates of Convergence for Nearest Neighbor Classification

Abstract

Nearest neighbor methods are a popular class of nonparametric estimators with several desirable properties, such as adaptivity to different distance scales in different regions of space. Prior work on convergence rates for nearest neighbor classification has not fully reflected these subtle properties. We analyze the behavior of these estimators in metric spaces and provide finite-sample, distribution-dependent rates of convergence under minimal assumptions. As a by-product, we are able to establish the universal consistency of nearest neighbor in a broader range of data spaces than was previously known. We illustrate our upper and lower bounds by introducing smoothness classes that are customized for nearest neighbor classification.

References in corpus (1)

• Fast learning rates for plug-in classifiers

Cited by in corpus (12)

• Class-Weighted Classification: Trade-offs and Robust Approaches
• An adaptive nearest neighbor rule for classification
• Minimax Rate Optimal Adaptive Nearest Neighbor Classification and Regression
• Under-bagging Nearest Neighbors for Imbalanced Classification
• Classification with unknown class-conditional label noise on non-compact feature spaces
• Deep k-NN for Noisy Labels
• Expressivity of expand-and-sparsify representations
• Multiclass Classification via Class-Weighted Nearest Neighbors
• Predictive Power of Nearest Neighbors Algorithm under Random Perturbation
• Distributed Nearest Neighbor Classification
• Nonparametric adaptive active learning under local smoothness condition
• Exploring Adversarial Examples for Efficient Active Learning in Machine Learning Classifiers