Tree Space Prototypes: Another Look at Making Tree Ensembles Interpretable

Abstract

Ensembles of decision trees perform well on many problems, but are not interpretable. In contrast to existing approaches in interpretability that focus on explaining relationships between features and predictions, we propose an alternative approach to interpret tree ensemble classifiers by surfacing representative points for each class -- prototypes. We introduce a new distance for Gradient Boosted Tree models, and propose new, adaptive prototype selection methods with theoretical guarantees, with the flexibility to choose a different number of prototypes in each class. We demonstrate our methods on random forests and gradient boosted trees, showing that the prototypes can perform as well as or even better than the original tree ensemble when used as a nearest-prototype classifier. In a user study, humans were better at predicting the output of a tree ensemble classifier when using prototypes than when using Shapley values, a popular feature attribution method. Hence, prototypes present a viable alternative to feature-based explanations for tree ensembles.

Camera-ready version for ACM-IMS FODS 2020. A short version was presented at NIPS 2016 Workshop on Interpretable Machine Learning for Complex Systems

References in corpus (8)

• A Unified Approach to Interpreting Model Predictions
• Towards A Rigorous Science of Interpretable Machine Learning
• Understanding Black-box Predictions via Influence Functions
• Model-Agnostic Interpretability of Machine Learning
• Prototype selection for interpretable classification
• Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions
• Making Tree Ensembles Interpretable
• Interpreting Models via Single Tree Approximation

Cited by in corpus (12)

• Local Rule-Based Explanations of Black Box Decision Systems
• Post-hoc explanation of black-box classifiers using confident itemsets
• Explainable Artificial Intelligence: a Systematic Review
• Concise Fuzzy System Modeling Integrating Soft Subspace Clustering and Sparse Learning
• Open the Black Box Data-Driven Explanation of Black Box Decision Systems
• Considerations When Learning Additive Explanations for Black-Box Models
• Born-Again Tree Ensembles
• Machine Guides, Human Supervises: Interactive Learning with Global Explanations
• Technologies for Trustworthy Machine Learning: A Survey in a Socio-Technical Context
• Why is the prediction wrong? Towards underfitting case explanation via meta-classification
• Toward Machine-Guided, Human-Initiated Explanatory Interactive Learning
• An Overview of Computational Approaches for Interpretation Analysis