Unbiased variable importance for random forests

Abstract

The default variable-importance measure in random Forests, Gini importance, has been shown to suffer from the bias of the underlying Gini-gain splitting criterion. While the alternative permutation importance is generally accepted as a reliable measure of variable importance, it is also computationally demanding and suffers from other shortcomings. We propose a simple solution to the misleading/untrustworthy Gini importance which can be viewed as an overfitting problem: we compute the loss reduction on the out-of-bag instead of the in-bag training samples.

References in corpus (6)

• Scikit-learn: Machine Learning in Python
• ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R
• Hyperparameters and Tuning Strategies for Random Forest
• A Debiased MDI Feature Importance Measure for Random Forests
• Unbiased variable importance for random forests
• Unbiased Measurement of Feature Importance in Tree-Based Methods

Cited by in corpus (6)

• Unbiased variable importance for random forests
• Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model, or There Is No Free Variable Importance
• Sequential Permutation Testing of Random Forest Variable Importance Measures
• Provable Boolean Interaction Recovery from Tree Ensemble obtained via Random Forests
• From unbiased MDI Feature Importance to Explainable AI for Trees
• Data-driven advice for interpreting local and global model predictions in bioinformatics problems