Asymptotic Distributions and Rates of Convergence for Random Forests via Generalized U-statistics
arXiv:1905.10651
Abstract
Random forests remain among the most popular off-the-shelf supervised learning algorithms. Despite their well-documented empirical success, however, until recently, few theoretical results were available to describe their performance and behavior. In this work we push beyond recent work on consistency and asymptotic normality by establishing rates of convergence for random forests and other supervised learning ensembles. We develop the notion of generalized U-statistics and show that within this framework, random forest predictions can potentially remain asymptotically normal for larger subsample sizes than previously established. We also provide Berry-Esseen bounds in order to quantify the rate at which this convergence occurs, making explicit the roles of the subsample size and the number of trees in determining the distribution of random forest predictions.
arXiv comment76 pages, 7 figure
References in corpus (12)
- Analysis of a Random Forests Model
- Consistency of random forests
- Normal approximation under local dependence
- Quantifying Uncertainty in Random Forests via Confidence Intervals and Hypothesis Tests
- Predicting the direction of stock market prices using random forest
- Approximating high-dimensional infinite-order -statistics: statistical and computational guarantees
- Consistency of Online Random Forests
- On Nesting Monte Carlo Estimators
- Making Sense of Random Forest Probabilities: a Kernel Perspective
- Scalable and Efficient Hypothesis Testing with Random Forests
- Getting Better from Worse: Augmented Bagging and a Cautionary Tale of Variable Importance
- Consistency of survival tree and forest models: splitting bias and correction
Cited by in corpus (9)
- Randomization as Regularization: A Degrees of Freedom Explanation for Random Forest Success
- Getting Better from Worse: Augmented Bagging and a Cautionary Tale of Variable Importance
- Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution
- -statistics and Variance Estimation
- Bias, Consistency, and Alternative Perspectives of the Infinitesimal Jackknife
- Random Planted Forest: a directly interpretable tree ensemble
- On the Subbagging Estimation for Massive Data
- Forest Guided Smoothing
- Locally Optimized Random Forests