Analysis of a Random Forests Model
arXiv:1005.0208
Abstract
Random forests are a scheme proposed by Leo Breiman in the 2000's for building a predictor ensemble with a set of decision trees that grow in randomly selected subspaces of data. Despite growing interest and practical use, there has been little exploration of the statistical properties of random forests, and little is known about the mathematical forces driving the algorithm. In this paper, we offer an in-depth analysis of a random forests model suggested by Breiman in \cite{Bre04}, which is very close to the original algorithm. We show in particular that the procedure is consistent and adapts to sparsity, in the sense that its rate of convergence depends only on the number of strong features and not on how many noise variables are present.
Cited by in corpus (52)
- Consistency of random forests
- A Parallel Random Forest Algorithm for Big Data in a Spark Cloud Computing Environment
- Narrowing the Gap: Random Forests In Theory and In Practice
- Quantifying Uncertainty in Random Forests via Confidence Intervals and Hypothesis Tests
- A Parallel Patient Treatment Time Prediction Algorithm and its Applications in Hospital Queuing-Recommendation in a Big Data Environment
- COBRA: A Combined Regression Strategy
- Consistency of Online Random Forests
- Asymptotic Theory for Random Forests
- Trees, forests, and impurity-based variable importance
- Selective inference for effect modification via the lasso
- A method for finding anomalous astronomical light curves and their analogs
- Classical and quantum regression analysis for the optoelectronic performance of NTCDA/p-Si UV photodiode
- Randomization as Regularization: A Degrees of Freedom Explanation for Random Forest Success
- Scalable and Efficient Hypothesis Testing with Random Forests
- Estimation and Inference with Trees and Forests in High Dimensions
- Provable Boolean Interaction Recovery from Tree Ensemble obtained via Random Forests
- Assisted Learning: A Framework for Multi-Organization Learning
- Estimating heterogeneous treatment effects with right-censored data via causal survival forests
- Consistency of survival tree and forest models: splitting bias and correction
- Asymptotic Distributions and Rates of Convergence for Random Forests via Generalized U-statistics
- Sharp Analysis of a Simple Model for Random Forests
- A Unified Framework for Random Forest Prediction Error Estimation
- Transformation Forests
- Asymptotic Properties of High-Dimensional Random Forests
- Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution
- Impact of subsampling and pruning on random forests
- On the Optimality of Trees Generated by ID3
- Learning-based Feedback Controller for Deformable Object Manipulation
- A Novel Machine Learning Approach to Disentangle Multi-Temperature Regions in Galaxy Clusters
- Cosmic topology. Part IVa. Classification of manifolds using machine learning: a case study with small toroidal universes
- Confidence Intervals for Random Forests: The Jackknife and the Infinitesimal Jackknife
- Banzhaf Random Forests
- Analyzing the tree-layer structure of Deep Forests
- Random forests and kernel methods
- On the use of Harrell's C for clinical risk prediction via random survival forests
- Comments on: "A Random Forest Guided Tour" by G. Biau and E. Scornet
- Multi-officer Routing for Patrolling High Risk Areas Jointly Learned from Check-ins, Crime and Incident Response Data
- Global and Local Two-Sample Tests via Regression
- Cloth Manipulation Using Random-Forest-Based Imitation Learning
- Continuous-Time Birth-Death MCMC for Bayesian Regression Tree Models
- Targeting predictors in random forest regression
- On the Consistency of a Random Forest Algorithm in the Presence of Missing Entries
- Random Planted Forest: a directly interpretable tree ensemble
- Deep Neural Networks Guided Ensemble Learning for Point Estimation
- MPBART - Multinomial Probit Bayesian Additive Regression Trees
- WildWood: a new Random Forest algorithm
- Modelling hetegeneous treatment effects by quantitle local polynomial decision tree and forest
- XtracTree: a Simple and Effective Method for Regulator Validation of Bagging Methods Used in Retail Banking
- PAC-Bayesian aggregation of affine estimators
- A Flexible Procedure for Mixture Proportion Estimation in Positive-Unlabeled Learning
- Estimating a sharp convergence bound for randomized ensembles
- Multinomial Random Forest: Toward Consistency and Privacy-Preservation