Hyperparameters and Tuning Strategies for Random Forest

Abstract

The random forest algorithm (RF) has several hyperparameters that have to be set by the user, e.g., the number of observations drawn randomly for each tree and whether they are drawn with or without replacement, the number of variables drawn randomly for each split, the splitting rule, the minimum number of samples that a node must contain and the number of trees. In this paper, we first provide a literature review on the parameters' influence on the prediction performance and on variable importance measures. It is well known that in most cases RF works reasonably well with the default values of the hyperparameters specified in software packages. Nevertheless, tuning the hyperparameters can improve the performance of RF. In the second part of this paper, after a brief overview of tuning strategies we demonstrate the application of one of the most established tuning strategies, model-based optimization (MBO). To make it easier to use, we provide the tuneRanger R package that tunes RF with MBO automatically. In a benchmark study on several datasets, we compare the prediction performance and runtime of tuneRanger with other tuning implementations in R and RF with default hyperparameters.

arXiv comment19 pages, 2 figures

References in corpus (8)

• ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R
• To Explain or to Predict?
• OpenML: networked science in machine learning
• Tunability: Importance of Hyperparameters of Machine Learning Algorithms
• Hyperparameter Importance Across Datasets
• Unbiased split variable selection for random survival forests using maximally selected rank statistics
• Random Forests: some methodological insights
• OpenML: An R Package to Connect to the Machine Learning Platform OpenML

Cited by in corpus (23)

• Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features
• Machine Learning-Enabled IoT Security: Open Issues and Challenges Under Advanced Persistent Threats
• Understanding overfitting in random forest for probability estimation: a visualization and simulation study
• Large-scale benchmark study of survival prediction methods using multi-omics data
• MechProNet: Machine Learning Prediction of Mechanical Properties in Metal Additive Manufacturing
• Review: Deep Learning in Electron Microscopy
• Unbiased variable importance for random forests
• Manifold embedding data-driven mechanics
• On Wasted Contributions: Understanding the Dynamics of Contributor-Abandoned Pull Requests
• Artificial Intelligence-Based Analytics for Impacts of COVID-19 and Online Learning on College Students' Mental Health
• A Comprehensive Study of Machine Learning Techniques for Log-Based Anomaly Detection
• Conceptual Views on Tree Ensemble Classifiers
• CovidAlert -- A Wristwatch-based System to Alert Users from Face Touching
• Analysis of the Evolution of Parametric Drivers of High-End Sea-Level Hazards
• Supervised topological data analysis for MALDI mass spectrometry imaging applications
• Active galactic nuclei catalog from the AKARI NEP Wide field
• What Makes Forest-Based Heterogeneous Treatment Effect Estimators Work?
• Data Distribution-based Curriculum Learning
• Learning to Select SAT Encodings for Pseudo-Boolean and Linear Integer Constraints
• Comprehensive and Spatially Detailed Passenger Vehicle and Truck Traffic Volume Data for the United States Estimated by Machine Learning
• Scalable Machine Learning Models for Predicting Quantum Transport in Disordered 2D Hexagonal Materials
• Observability of an ultraheavy diquark decaying into vectorlike quarks at the LHC
• DRAGNs in the Forest: Identifying Artifacts with Random Forest Models in the VLASS DRAGNs Catalog