Local Rule-Based Explanations of Black Box Decision Systems

Abstract

The recent years have witnessed the rise of accurate but obscure decision systems which hide the logic of their internal decision processes to the users. The lack of explanations for the decisions of black box systems is a key ethical issue, and a limitation to the adoption of machine learning components in socially sensitive and safety-critical contexts. %Therefore, we need explanations that reveals the reasons why a predictor takes a certain decision. In this paper we focus on the problem of black box outcome explanation, i.e., explaining the reasons of the decision taken on a specific instance. We propose LORE, an agnostic method able to provide interpretable and faithful explanations. LORE first leans a local interpretable predictor on a synthetic neighborhood generated by a genetic algorithm. Then it derives from the logic of the local interpretable predictor a meaningful explanation consisting of: a decision rule, which explains the reasons of the decision; and a set of counterfactual rules, suggesting the changes in the instance's features that lead to a different outcome. Wide experiments show that LORE outperforms existing methods and baselines both in the quality of explanations and in the accuracy in mimicking the black box.

References in corpus (5)

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Towards A Rigorous Science of Interpretable Machine Learning
• Making Tree Ensembles Interpretable
• Tree Space Prototypes: Another Look at Making Tree Ensembles Interpretable
• Programs as Black-Box Explanations

Cited by in corpus (60)

• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations
• A Survey on the Explainability of Supervised Machine Learning
• Post-hoc explanation of black-box classifiers using confident itemsets
• Explainable Matrix -- Visualization for Global and Local Interpretability of Random Forest Classification Ensembles
• Deep Learning for Android Malware Defenses: a Systematic Literature Review
• Logic Explained Networks
• Explainable Artificial Intelligence for Process Mining: A General Overview and Application of a Novel Local Explanation Approach for Predictive Process Monitoring
• Algorithmic Recourse: from Counterfactual Explanations to Interventions
• An Interpretable Model with Globally Consistent Explanations for Credit Risk
• Counterfactual Shapley Additive Explanations
• Entropy-based Logic Explanations of Neural Networks
• Ethics-Based Auditing of Automated Decision-Making Systems: Intervention Points and Policy Implications
• Testing and verification of neural-network-based safety-critical control software: A systematic literature review
• On Interpretability of Artificial Neural Networks: A Survey
• A survey of algorithmic recourse: definitions, formulations, solutions, and prospects
• Causality Learning: A New Perspective for Interpretable Machine Learning
• Open the Black Box Data-Driven Explanation of Black Box Decision Systems
• MoËT: Mixture of Expert Trees and its Application to Verifiable Reinforcement Learning
• Issues with post-hoc counterfactual explanations: a discussion
• Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications
• Individual Explanations in Machine Learning Models: A Survey for Practitioners
• Why model why? Assessing the strengths and limitations of LIME
• MultiVerse: Causal Reasoning using Importance Sampling in Probabilistic Programming
• Efficient Learning of Interpretable Classification Rules
• Synthesizing explainable counterfactual policies for algorithmic recourse with program synthesis
• Efficient computation of counterfactual explanations of LVQ models
• Certification of embedded systems based on Machine Learning: A survey
• A multi-component framework for the analysis and design of explainable artificial intelligence
• Consistent Counterfactuals for Deep Models
• An exact counterfactual-example-based approach to tree-ensemble models interpretability
• Explaining Predictions by Approximating the Local Decision Boundary
• Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?
• Can We Trust Your Explanations? Sanity Checks for Interpreters in Android Malware Analysis
• Semantic XAI for contextualized demand forecasting explanations
• A Review of Explainable Artificial Intelligence in Manufacturing
• On the overlooked issue of defining explanation objectives for local-surrogate explainers
• On Locality of Local Explanation Models
• YASENN: Explaining Neural Networks via Partitioning Activation Sequences
• Model-agnostic and Scalable Counterfactual Explanations via Reinforcement Learning
• Understanding surrogate explanations: the interplay between complexity, fidelity and coverage
• Interpretable Companions for Black-Box Models
• Explaining the Behavior of Black-Box Prediction Algorithms with Causal Learning
• Shapley Explanation Networks
• Visualizing Rule Sets: Exploration and Validation of a Design Space
• Efficient Decompositional Rule Extraction for Deep Neural Networks
• Beyond Average Performance -- exploring regions of deviating performance for black box classification models
• Bandits for Learning to Explain from Explanations
• Towards Interpretable ANNs: An Exact Transformation to Multi-Class Multivariate Decision Trees
• Accurate and Intuitive Contextual Explanations using Linear Model Trees
• Interpreting Undesirable Pixels for Image Classification on Black-Box Models
• Visualizing Color-wise Saliency of Black-Box Image Classification Models
• LioNets: Local Interpretation of Neural Networks through Penultimate Layer Decoding
• ACE -- An Anomaly Contribution Explainer for Cyber-Security Applications
• Identifying the Most Explainable Classifier
• When Stability meets Sufficiency: Informative Explanations that do not Overwhelm
• Automated Dependence Plots
• An Exploration And Validation of Visual Factors in Understanding Classification Rule Sets
• An Active Approach for Model Interpretation
• XPROAX-Local explanations for text classification with progressive neighborhood approximation