Analysis of purely random forests bias

Abstract

Random forests are a very effective and commonly used statistical method, but their full theoretical analysis is still an open problem. As a first step, simplified models such as purely random forests have been introduced, in order to shed light on the good performance of random forests. In this paper, we study the approximation error (the bias) of some purely random forest models in a regression framework, focusing in particular on the influence of the number of trees in the forest. Under some regularity assumptions on the regression function, we show that the bias of an infinite forest decreases at a faster rate (with respect to the size of each tree) than a single tree. As a consequence, infinite forests attain a strictly better risk rate (with respect to the sample size) than single trees. Furthermore, our results allow to derive a minimum number of trees sufficient to reach the same rate as an infinite forest. As a by-product of our analysis, we also show a link between the bias of purely random forests and the bias of some kernel estimators.

References in corpus (1)

• V-fold cross-validation improved: V-fold penalization

Cited by in corpus (18)

• Estimation and Inference with Trees and Forests in High Dimensions
• Estimating heterogeneous treatment effects with right-censored data via causal survival forests
• Consistency of survival tree and forest models: splitting bias and correction
• Sharp Analysis of a Simple Model for Random Forests
• Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution
• Impact of subsampling and pruning on random forests
• A Learning Theoretic Perspective on Local Explainability
• Bayesian Nonparametric Space Partitions: A Survey
• Random forests and kernel methods
• On the use of Harrell's C for clinical risk prediction via random survival forests
• Universal consistency and minimax rates for online Mondrian Forests
• Comments on: "A Random Forest Guided Tour" by G. Biau and E. Scornet
• Estimating the Algorithmic Variance of Randomized Ensembles via the Bootstrap
• WildWood: a new Random Forest algorithm
• Best-scored Random Forest Classification
• Online Binary Space Partitioning Forests
• Censored Quantile Regression Forests
• Is interpolation benign for random forest regression?