Predictive learning via rule ensembles

Abstract

General regression and classification models are constructed as linear combinations of simple rules derived from the data. Each rule consists of a conjunction of a small number of simple statements concerning the values of individual input variables. These rule ensembles are shown to produce predictive accuracy comparable to the best methods. However, their principal advantage lies in interpretation. Because of its simple form, each rule is easy to understand, as is its influence on individual predictions, selected subsets of predictions, or globally over the entire space of joint input variable values. Similarly, the degree of relevance of the respective input variables can be assessed globally, locally in different regions of the input space, or at individual prediction points. Techniques are presented for automatically identifying those variables that are involved in interactions with other variables, the strength and degree of those interactions, as well as the identities of the other variables with which they interact. Graphical representations are used to visualize both main and interaction effects.

Published in at http://dx.doi.org/10.1214/07-AOAS148 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Cited by in corpus (115)

• Interpretable machine learning: definitions, methods, and applications
• A Survey on the Explainability of Supervised Machine Learning
• The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• TMVA - Toolkit for Multivariate Data Analysis
• Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges
• Big-Data Science in Porous Materials: Materials Genomics and Machine Learning
• Iterative Random Forests to detect predictive and stable high-order interactions
• Observation of Single Top-Quark Production
• Towards Out-Of-Distribution Generalization: A Survey
• Machine learning algorithms to infer trait-matching and predict species interactions in ecological networks
• The impact of feature importance methods on the interpretation of defect classifiers
• FairSight: Visual Analytics for Fairness in Decision Making
• Expert-Augmented Machine Learning
• Node harvest
• How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanations
• Explainable Artificial Intelligence Approaches: A Survey
• Rule Extraction in Unsupervised Anomaly Detection for Model Explainability: Application to OneClass SVM
• "Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations
• A Simple and Effective Model-Based Variable Importance Measure
• Model-independent measurement of $\mathbf{t}$-channel single top quark production in $\mathbf{p}\bar{\mathbf{p}}$ collisions at $\sqrt{\mathbf{s}}=1.96$ TeV
• Augmenting Interpretable Models with LLMs during Training
• Feature Selection via Regularized Trees
• Interpreting Tree Ensembles with inTrees
• Interpretable Random Forests via Rule Extraction
• Explainable Artificial Intelligence (XAI) for Malware Analysis: A Survey of Techniques, Applications, and Open Challenges
• Measurements of single top quark production cross sections and |Vtb| in ppbar collisions at sqrt{s}=1.96 TeV
• Locally Interpretable Models and Effects based on Supervised Partitioning (LIME-SUP)
• GuideR: a guided separate-and-conquer rule learning in classification, regression, and survival settings
• MLIC: A MaxSAT-Based framework for learning interpretable classification rules
• Fitting Prediction Rule Ensembles with R Package pre
• How does this interaction affect me? Interpretable attribution for feature interactions
• Considerations When Learning Additive Explanations for Black-Box Models
• Detecting Statistical Interactions from Neural Network Weights
• Model Distillation for Revenue Optimization: Interpretable Personalized Pricing
• Derivative based global sensitivity measures
• DMiner: Dashboard Design Mining and Recommendation
• Counterfactual Phenotyping with Censored Time-to-Events
• Interpretable machine learning applied to on-farm biosecurity and porcine reproductive and respiratory syndrome virus
• Learning Security Classifiers with Verified Global Robustness Properties
• Born-Again Tree Ensembles
• Stabilized Sparse Ordinal Regression for Medical Risk Stratification
• Interpretable Representations in Explainable AI: From Theory to Practice
• Tree based machine learning framework for predicting ground state energies of molecules
• Rule extraction based on extreme learning machine and an improved ant-miner algorithm for transient stability assessment
• GAMI-Net: An Explainable Neural Network based on Generalized Additive Models with Structured Interactions
• Generalized Linear Rule Models
• Observation of Single Top Quark Production at the Tevatron Collider
• An Introduction to Artificial Prediction Markets for Classification
• What Did You Think Would Happen? Explaining Agent Behaviour Through Intended Outcomes
• Signed iterative random forests to identify enhancer-associated transcription factor binding
• Learning Nonlinear Functions Using Regularized Greedy Forest
• Explainable AI for Correct Root Cause Analysis of Product Quality in Injection Moulding
• Local Multi-Label Explanations for Random Forest
• Disentangled Attribution Curves for Interpreting Random Forests and Boosted Trees
• Safe RuleFit: Learning Optimal Sparse Rule Model by Meta Safe Screening
• VINE: Visualizing Statistical Interactions in Black Box Models
• Interpretable and Fair Boolean Rule Sets via Column Generation
• From Point to probabilistic gradient boosting for claim frequency and severity prediction
• Bias, Fairness, and Accountability with AI and ML Algorithms
• Learning Interpretable Decision Rule Sets: A Submodular Optimization Approach
• Scalable Bayesian Rule Lists
• Rationalizing Predictions by Adversarial Information Calibration
• Explainable Deep Behavioral Sequence Clustering for Transaction Fraud Detection
• Application of Machine Learning in Rock Facies Classification with Physics-Motivated Feature Augmentation
• Observation of Single Top Quark Production
• Applying Rule Ensembles to the Search for Super-Symmetry at the Large Hadron Collider
• Information Fusion via Symbolic Regression: A Tutorial in the Context of Human Health
• FIRE: An Optimization Approach for Fast Interpretable Rule Extraction
• Explaining Predictions by Characteristic Rules
• Model Interpretation: A Unified Derivative-based Framework for Nonparametric Regression and Supervised Machine Learning
• Random Forests for dependent data
• Generating Explainable Rule Sets from Tree-Ensemble Learning Methods by Answer Set Programming
• LIBRE: Learning Interpretable Boolean Rule Ensembles
• Towards interpreting ML-based automated malware detection models: a survey
• Detection of Interacting Variables for Generalized Linear Models via Neural Networks
• Rule Covering for Interpretation and Boosting
• Towards Rigorous Interpretations: a Formalisation of Feature Attribution
• Computer-Assisted Creation of Boolean Search Rules for Text Classification in the Legal Domain
• Survey on AI Ethics: A Socio-technical Perspective
• Dimension Reduction Using Rule Ensemble Machine Learning Methods: A Numerical Study of Three Ensemble Methods
• A cautionary tale on fitting decision trees to data from additive models: generalization lower bounds
• Interpret-able feedback for AutoML systems
• Iterated Local Search with Linkage Learning
• Lossless (and Lossy) Compression of Random Forests
• Gene selection with guided regularized random forest
• Towards Interaction Detection Using Topological Analysis on Neural Networks
• Improved prediction rule ensembling through model-based data generation
• Uncertainty-aware Sensitivity Analysis Using Rényi Divergences
• Rule-Mining based classification: a benchmark study
• Generating Global and Local Explanations for Tree-Ensemble Learning Methods by Answer Set Programming
• Group Heterogeneity Assessment for Multilevel Models
• Interpretabilité des modèles : état des lieux des méthodes et application à l'assurance
• Random Intersection Trees
• Soft Rule Ensembles for Statistical Learning
• Dynamic Interpretability for Model Comparison via Decision Rules
• Modern approaches to building interpretable models of the property market using machine learning on the base of mass cadastral valuation
• Interpretable self-driving sputter epitaxy: from black-box optimization to human-usable growth rules
• SplitWise Regression: Stepwise Modeling with Adaptive Dummy Encoding
• On the Use of Interpretable Machine Learning for the Management of Data Quality
• Constrained Co-evolutionary Metamorphic Differential Testing for Autonomous Systems with an Interpretability Approach
• Beyond Discriminant Patterns: On the Robustness of Decision Rule Ensembles
• Interaction as Interference: A Quantum-Inspired Aggregation Approach
• XPROAX-Local explanations for text classification with progressive neighborhood approximation
• Interpretability of Blackbox Machine Learning Models through Dataview Extraction and Shadow Model creation
• A Clustering and Demotion Based Algorithm for Inductive Learning of Default Theories
• Nonparametric Functional Approximation with Delaunay Triangulation
• Near-axis quasi-isodynamic database
• Mechanistic Interpretation of Machine Learning Inference: A Fuzzy Feature Importance Fusion Approach
• Interpretable Artificial Intelligence through the Lens of Feature Interaction
• Achieving Transparency Report Privacy in Linear Time
• DA-DGCEx: Ensuring Validity of Deep Guided Counterfactual Explanations With Distribution-Aware Autoencoder Loss
• Explaining Anomalies in Groups with Characterizing Subspace Rules
• Online Interaction Detection for Click-Through Rate Prediction
• Induction of Non-Monotonic Logic Programs to Explain Boosted Tree Models Using LIME