Explainable AI for Trees: From Local Explanations to Global Understanding

Abstract

Tree-based machine learning models such as random forests, decision trees, and gradient boosted trees are the most popular non-linear predictive models used in practice today, yet comparatively little attention has been paid to explaining their predictions. Here we significantly improve the interpretability of tree-based models through three main contributions: 1) The first polynomial time algorithm to compute optimal explanations based on game theory. 2) A new type of explanation that directly measures local feature interaction effects. 3) A new set of tools for understanding global model structure based on combining many local explanations of each prediction. We apply these tools to three medical machine learning problems and show how combining many high-quality local explanations allows us to represent global structure while retaining local faithfulness to the original model. These tools enable us to i) identify high magnitude but low frequency non-linear mortality risk factors in the general US population, ii) highlight distinct population sub-groups with shared risk characteristics, iii) identify non-linear interaction effects among risk factors for chronic kidney disease, and iv) monitor a machine learning model deployed in a hospital by identifying which features are degrading the model's performance over time. Given the popularity of tree-based machine learning models, these improvements to their interpretability have implications across a broad set of domains.

References in corpus (6)

• Scikit-learn: Machine Learning in Python
• Striving for Simplicity: The All Convolutional Net
• DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Network
• How to Explain Individual Classification Decisions
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• Model Agnostic Supervised Local Explanations

Cited by in corpus (21)

• Relationships between distortions of inorganic framework and band gap of layered hybrid halide perovskites
• Fast TreeSHAP: Accelerating SHAP Value Computation for Trees
• Interpreting a Recurrent Neural Network's Predictions of ICU Mortality Risk
• Explainable Machine Learning for Predicting Homicide Clearance in the United States
• Explainable artificial intelligence model to predict acute critical illness from electronic health records
• Detecting new obfuscated malware variants: A lightweight and interpretable machine learning approach
• Evaluating the Correctness of Explainable AI Algorithms for Classification
• Uncertainty Quantification for Data-driven Turbulence Modelling with Mondrian Forests
• Efficient learning of large sets of locally optimal classification rules
• Best of both worlds: local and global explanations with human-understandable concepts
• Generalized SHAP: Generating multiple types of explanations in machine learning
• Pitfalls of Explainable ML: An Industry Perspective
• Negation Detection for Clinical Text Mining in Russian
• Explaining Deep Learning Models for Structured Data using Layer-Wise Relevance Propagation
• A Framework for Democratizing AI
• LionForests: Local Interpretation of Random Forests
• A framework for predicting, interpreting, and improving Learning Outcomes
• Towards Explaining STEM Document Classification using Mathematical Entity Linking
• Activity Detection from Encrypted Remote Desktop Protocol Traffic
• Feature Synergy, Redundancy, and Independence in Global Model Explanations using SHAP Vector Decomposition
• A gradient boosting approach for optimal selection of bidding strategies in reservoir hydro