Explaining Explanations in AI

Abstract

Recent work on interpretability in machine learning and AI has focused on the building of simplified models that approximate the true criteria used to make decisions. These models are a useful pedagogical device for teaching trained professionals how to predict what decisions will be made by the complex system, and most importantly how the system might break. However, when considering any such model it's important to remember Box's maxim that "All models are wrong but some are useful." We focus on the distinction between these models and explanations in philosophy and sociology. These models can be understood as a "do it yourself kit" for explanations, allowing a practitioner to directly answer "what if questions" or generate contrastive explanations without external assistance. Although a valuable ability, giving these models as explanations appears more difficult than necessary, and other forms of explanation may not have the same trade-offs. We contrast the different schools of thought on what makes an explanation, and suggest that machine learning might benefit from viewing the problem more broadly.

arXiv commentFAT* 2019 Proceedings

References in corpus (15)

• Explaining and Harnessing Adversarial Examples
• A Unified Approach to Interpreting Model Predictions
• Methods for Interpreting and Understanding Deep Neural Networks
• Learning Important Features Through Propagating Activation Differences
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• How to Explain Individual Classification Decisions
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences
• Interpretable Classification Models for Recidivism Prediction
• Tackling the Algorithmic Control Crisis -- the Technical, Legal, and Ethical Challenges of Research into Algorithmic Agents
• The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification
• Interpretability via Model Extraction
• Interpretable & Explorable Approximations of Black Box Models
• High Dimensional Spaces, Deep Learning and Adversarial Examples

Cited by in corpus (96)

• The Ethics of AI Ethics -- An Evaluation of Guidelines
• Explainable Machine Learning for Scientific Insights and Discoveries
• Deep Neural Networks and Tabular Data: A Survey
• The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies
• What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research
• Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions
• Expanding Explainability: Towards Social Transparency in AI systems
• A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?
• "Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
• Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
• Explainable Artificial Intelligence and Machine Learning: A reality rooted perspective
• Interactive Model Cards: A Human-Centered Approach to Model Documentation
• A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability
• Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI
• Explainable Artificial Intelligence for Process Mining: A General Overview and Application of a Novel Local Explanation Approach for Predictive Process Monitoring
• XAIR: A Framework of Explainable AI in Augmented Reality
• The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanations
• Comparing the Perceived Legitimacy of Content Moderation Processes: Contractors, Algorithms, Expert Panels, and Digital Juries
• How Case Based Reasoning Explained Neural Networks: An XAI Survey of Post-Hoc Explanation-by-Example in ANN-CBR Twins
• POTs: Protective Optimization Technologies
• Rationalization for Explainable NLP: A Survey
• Generating Counterfactual and Contrastive Explanations using SHAP
• Measurable Counterfactual Local Explanations for Any Classifier
• The Conflict Between Explainable and Accountable Decision-Making Algorithms
• The Impact of Explanations on Layperson Trust in Artificial Intelligence-Driven Symptom Checker Apps: Experimental Study
• Mediating Community-AI Interaction through Situated Explanation: The Case of AI-Led Moderation
• A survey of algorithmic recourse: definitions, formulations, solutions, and prospects
• Ethical behavior in humans and machines -- Evaluating training data quality for beneficial machine learning
• IEEE 7010: A New Standard for Assessing the Well-being Implications of Artificial Intelligence
• Survey of XAI in digital pathology
• Bridging the Transparency Gap: What Can Explainable AI Learn From the AI Act?
• Unsupervised Concept Drift Detection from Deep Learning Representations in Real-time
• One vs. Many: Comprehending Accurate Information from Multiple Erroneous and Inconsistent AI Generations
• Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learning
• Global Aggregations of Local Explanations for Black Box models
• Shapley Flow: A Graph-based Approach to Interpreting Model Predictions
• Designing Accountable Systems
• Explaining Naive Bayes and Other Linear Classifiers with Polynomial Time and Delay
• Interpretable Representations in Explainable AI: From Theory to Practice
• Bias in Data-driven AI Systems -- An Introductory Survey
• "The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep Learning
• Question-Driven Design Process for Explainable AI User Experiences
• GAM Coach: Towards Interactive and User-centered Algorithmic Recourse
• Ask to Know More: Generating Counterfactual Explanations for Fake Claims
• Foundations of Explainable Knowledge-Enabled Systems
• ExplainitAI: When do we trust artificial intelligence? The influence of content and explainability in a cross-cultural comparison
• Explaining Any ML Model? -- On Goals and Capabilities of XAI
• GCExplainer: Human-in-the-Loop Concept-based Explanations for Graph Neural Networks
• The Sanction of Authority: Promoting Public Trust in AI
• Beyond Model Interpretability: Socio-Structural Explanations in Machine Learning
• The AI-DEC: A Card-based Design Method for User-centered AI Explanations
• On Relating 'Why?' and 'Why Not?' Explanations
• CARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation Algorithms
• Unexplainability and Incomprehensibility of Artificial Intelligence
• Learn to Explain Efficiently via Neural Logic Inductive Learning
• From Spoken Thoughts to Automated Driving Commentary: Predicting and Explaining Intelligent Vehicles' Actions
• Efficient Explanations for Knowledge Compilation Languages
• Do Transformer Attention Heads Provide Transparency in Abstractive Summarization?
• E Pluribus Unum Interpretable Convolutional Neural Networks
• LIMEtree: Consistent and Faithful Surrogate Explanations of Multiple Classes
• Explaining Deep Learning Models for Structured Data using Layer-Wise Relevance Propagation
• A Visual Analytics Framework for Adversarial Text Generation
• Efficient Search for Diverse Coherent Explanations
• An exact counterfactual-example-based approach to tree-ensemble models interpretability
• Complementary reinforcement learning towards explainable agents
• Can We Trust Your Explanations? Sanity Checks for Interpreters in Android Malware Analysis
• A three-Level Framework for LLM-Enhanced eXplainable AI: From technical explanations to natural language
• From Shallow to Deep Interactions Between Knowledge Representation, Reasoning and Machine Learning (Kay R. Amel group)
• AI-Mediated Exchange Theory
• Challenging common interpretability assumptions in feature attribution explanations
• NLPGuard: A Framework for Mitigating the Use of Protected Attributes by NLP Classifiers
• Explainability in AI Based Applications: A Framework for Comparing Different Techniques
• Explaining classifiers to understand coarse-grained models
• Seven challenges for harmonizing explainability requirements
• Explainability Case Studies
• A Causal Lens for Peeking into Black Box Predictive Models: Predictive Model Interpretation via Causal Attribution
• Expressing Accountability Patterns using Structural Causal Models
• MedicalPatchNet: A Patch-Based Self-Explainable AI Architecture for Chest X-ray Classification
• Monitoring Trust in Human-Machine Interactions for Public Sector Applications
• Who is this Explanation for? Human Intelligence and Knowledge Graphs for eXplainable AI
• On Two XAI Cultures: A Case Study of Non-technical Explanations in Deployed AI System
• Accounts, Accountability and Agency for Safe and Ethical AI
• Limitations of Deep Neural Networks: a discussion of G. Marcus' critical appraisal of deep learning
• Explanatory Pluralism in Explainable AI
• Economics of Human-AI Ecosystem: Value Bias and Lost Utility in Multi-Dimensional Gaps
• One-shot Policy Elicitation via Semantic Reward Manipulation
• Deep Learning Based Decision Support for Medicine -- A Case Study on Skin Cancer Diagnosis
• Faithful and Plausible Explanations of Medical Code Predictions
• ALICE: Active Learning with Contrastive Natural Language Explanations
• Leveraging Clinical Context for User-Centered Explainability: A Diabetes Use Case
• Explaining Creative Artifacts
• A Series of Unfortunate Counterfactual Events: the Role of Time in Counterfactual Explanations
• A general framework for scientifically inspired explanations in AI
• Trustworthy Transparency by Design
• Towards Transparent Robotic Planning via Contrastive Explanations
• Longitudinal Distance: Towards Accountable Instance Attribution