Narrowing the Gap: Random Forests In Theory and In Practice

Abstract

Despite widespread interest and practical use, the theoretical properties of random forests are still not well understood. In this paper we contribute to this understanding in two ways. We present a new theoretically tractable variant of random regression forests and prove that our algorithm is consistent. We also provide an empirical evaluation, comparing our algorithm and other theoretically tractable random forest models to the random forest algorithm used in practice. Our experiments provide insight into the relative importance of different simplifications that theoreticians have made to obtain tractable models for analysis.

Under review by the International Conference on Machine Learning (ICML) 2014

References in corpus (1)

• Risk bounds for purely uniformly random forests

Cited by in corpus (24)

• Quantifying Uncertainty in Random Forests via Confidence Intervals and Hypothesis Tests
• Adaptive Concentration of Regression Trees, with Application to Random Forests
• Combining Static and Dynamic Features for Multivariate Sequence Classification
• Asymptotic Theory for Random Forests
• Heteroscedastic Treed Bayesian Optimisation
• An Empirical Analysis of Deep Learning for Cardinality Estimation
• Differentially- and non-differentially-private random decision trees
• Estimation and Inference with Trees and Forests in High Dimensions
• Unbiased Measurement of Feature Importance in Tree-Based Methods
• Provable Boolean Interaction Recovery from Tree Ensemble obtained via Random Forests
• Rectified Decision Trees: Towards Interpretability, Compression and Empirical Soundness
• Sharp Analysis of a Simple Model for Random Forests
• Revocable Federated Learning: A Benchmark of Federated Forest
• On the Optimality of Trees Generated by ID3
• Random Forests for Adaptive Nearest Neighbor Estimation of Information-Theoretic Quantities
• Muddling Label Regularization: Deep Learning for Tabular Datasets
• Banzhaf Random Forests
• Adaptive Random SubSpace Learning (RSSL) Algorithm for Prediction
• There is no Double-Descent in Random Forests
• Cost-complexity pruning of random forests
• Censored Quantile Regression Forests
• Multinomial Random Forest: Toward Consistency and Privacy-Preservation
• Function space analysis of deep learning representation layers
• Improving the Accuracy-Memory Trade-Off of Random Forests Via Leaf-Refinement