Mondrian Forests: Efficient Online Random Forests

Abstract

Ensembles of randomized decision trees, usually referred to as random forests, are widely used for classification and regression tasks in machine learning and statistics. Random forests achieve competitive predictive performance and are computationally efficient to train and test, making them excellent candidates for real-world prediction tasks. The most popular random forest variants (such as Breiman's random forest and extremely randomized trees) operate on batches of training data. Online methods are now in greater demand. Existing online random forests, however, require more training data than their batch counterpart to achieve comparable predictive performance. In this work, we use Mondrian processes (Roy and Teh, 2009) to construct ensembles of random decision trees we call Mondrian forests. Mondrian forests can be grown in an incremental/online fashion and remarkably, the distribution of online Mondrian forests is the same as that of batch Mondrian forests. Mondrian forests achieve competitive predictive performance comparable with existing online random forests and periodically re-trained batch random forests, while being more than an order of magnitude faster, thus representing a better computation vs accuracy tradeoff.

References in corpus (3)

• Scikit-learn: Machine Learning in Python
• Consistency of Online Random Forests
• Top-down particle filtering for Bayesian decision trees

Cited by in corpus (27)

• Active learning for data streams: a survey
• Efficient non-greedy optimization of decision trees
• Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes
• Isolation Mondrian Forest for Batch and Online Anomaly Detection
• A Machine Learning Approach to Online Fault Classification in HPC Systems
• Uncertainty Quantification for Data-driven Turbulence Modelling with Mondrian Forests
• The Case for Retraining of ML Models for IoT Device Identification at the Edge
• Mondrian Forests for Large-Scale Regression when Uncertainty Matters
• Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages
• Binary Space Partitioning Forests
• On the asymptotics of random forests
• Particle Gibbs for Bayesian Additive Regression Trees
• The Mondrian Process for Machine Learning
• Bayesian Nonparametric Space Partitions: A Survey
• Learning Interpretable Characteristic Kernels via Decision Forests
• Improving fraud prediction with incremental data balancing technique for massive data streams
• BEDS-Bench: Behavior of EHR-models under Distributional Shift--A Benchmark
• Non-Parametric Calibration for Classification
• Grading of Mammalian Cumulus Oocyte Complexes using Machine Learning for in Vitro Embryo Culture
• Online Binary Space Partitioning Forests
• WildWood: a new Random Forest algorithm
• Cross-project Classification of Security-related Requirements
• Interpretable Anomaly Detection with Mondrian P{ó}lya Forests on Data Streams
• Client-Based Intelligence for Resource Efficient Vehicular Big Data Transfer in Future 6G Network
• Selective Harvesting over Networks
• Wasserstein Random Forests and Applications in Heterogeneous Treatment Effects
• Shape Modeling with Spline Partitions