Consistent feature attribution for tree ensembles

Abstract

Note that a newer expanded version of this paper is now available at: arXiv:1802.03888 It is critical in many applications to understand what features are important for a model, and why individual predictions were made. For tree ensemble methods these questions are usually answered by attributing importance values to input features, either globally or for a single prediction. Here we show that current feature attribution methods are inconsistent, which means changing the model to rely more on a given feature can actually decrease the importance assigned to that feature. To address this problem we develop fast exact solutions for SHAP (SHapley Additive exPlanation) values, which were recently shown to be the unique additive feature attribution method based on conditional expectations that is both consistent and locally accurate. We integrate these improvements into the latest version of XGBoost, demonstrate the inconsistencies of current methods, and show how using SHAP values results in significantly improved supervised clustering performance. Feature importance values are a key part of understanding widely used models such as gradient boosting trees and random forests, so improvements to them have broad practical implications.

presented at 2017 ICML Workshop on Human Interpretability in Machine Learning (WHI 2017), Sydney, NSW, Australia

References in corpus (2)

• A Unified Approach to Interpreting Model Predictions
• An unexpected unity among methods for interpreting model predictions

Cited by in corpus (18)

• Boosting the performance of anomalous diffusion classifiers with the proper choice of features
• Shedding Light on Black Box Machine Learning Algorithms: Development of an Axiomatic Framework to Assess the Quality of Methods that Explain Individual Predictions
• On the Art and Science of Machine Learning Explanations
• Identification, mass and age of primary red clump stars from spectral features derived with the LAMOST DR7
• Synthetic Benchmarks for Scientific Research in Explainable Machine Learning
• Interpreting Black-box Machine Learning Models for High Dimensional Datasets
• A Unified Approach to Interpreting and Boosting Adversarial Transferability
• Enhancing operations management through smart sensors: measuring and improving well-being, interaction and performance of logistics workers
• Feature construction using explanations of individual predictions
• Your Face Mirrors Your Deepest Beliefs-Predicting Personality and Morals through Facial Emotion Recognition
• RUMBoost: Gradient Boosted Random Utility Models
• Anesthesiologist-level forecasting of hypoxemia with only SpO2 data using deep learning
• Importance measures derived from random forests: characterisation and extension
• Multi-objective optimization and explanation for stroke risk assessment in Shanxi province
• Explicating feature contribution using Random Forest proximity distances
• Early prediction of respiratory failure in the intensive care unit
• An exploration of the influence of path choice in game-theoretic attribution algorithms
• Interpreting Attributions and Interactions of Adversarial Attacks