Making Sense of Random Forest Probabilities: a Kernel Perspective

Abstract

A random forest is a popular tool for estimating probabilities in machine learning classification tasks. However, the means by which this is accomplished is unprincipled: one simply counts the fraction of trees in a forest that vote for a certain class. In this paper, we forge a connection between random forests and kernel regression. This places random forest probability estimation on more sound statistical footing. As part of our investigation, we develop a model for the proximity kernel and relate it to the geometry and sparsity of the estimation problem. We also provide intuition and recommendations for tuning a random forest to improve its probability estimates.

References in corpus (1)

• Consistency of random forests

Cited by in corpus (4)

• Randomization as Regularization: A Degrees of Freedom Explanation for Random Forest Success
• Distributional Generalization: A New Kind of Generalization
• Asymptotic Distributions and Rates of Convergence for Random Forests via Generalized U-statistics
• A new interval-based aggregation approach based on bagging and Interval Agreement Approach (IAA) in ensemble learning