Model-Agnostic Interpretability of Machine Learning

Abstract

Understanding why machine learning models behave the way they do empowers both system designers and end-users in many ways: in model selection, feature engineering, in order to trust and act upon the predictions, and in more intuitive user interfaces. Thus, interpretability has become a vital concern in machine learning, and work in the area of interpretable models has found renewed interest. In some applications, such models are as accurate as non-interpretable ones, and thus are preferred for their transparency. Even when they are not accurate, they may still be preferred when interpretability is of paramount importance. However, restricting machine learning to interpretable models is often a severe limitation. In this paper we argue for explaining machine learning predictions using model-agnostic approaches. By treating the machine learning models as black-box functions, these approaches provide crucial flexibility in the choice of models, explanations, and representations, improving debugging, comparison, and interfaces for a variety of users and models. We also outline the main challenges for such methods, and review a recently-introduced model-agnostic explanation approach (LIME) that addresses these challenges.

presented at 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), New York, NY

References in corpus (4)

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• Supersparse Linear Integer Models for Optimized Medical Scoring Systems
• Towards Universal Paraphrastic Sentence Embeddings

Cited by in corpus (99)

• Axiomatic Attribution for Deep Networks
• A Survey on the Explainability of Supervised Machine Learning
• EEG based Emotion Recognition: A Tutorial and Review
• Interpretability of machine learning based prediction models in healthcare
• Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches
• Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
• Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction
• The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations
• Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects
• Explainable, Interpretable & Trustworthy AI for Intelligent Digital Twin: Case Study on Remaining Useful Life
• Levels of explainable artificial intelligence for human-aligned conversational explanations
• TimeSHAP: Explaining Recurrent Models through Sequence Perturbations
• Explaining Classifiers with Causal Concept Effect (CaCE)
• Explainable Artificial Intelligence: a Systematic Review
• Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion
• To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods
• Towards Automatic Concept-based Explanations
• Logic Explained Networks
• On the Explanation of Machine Learning Predictions in Clinical Gait Analysis
• Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems
• Quantifying Model Complexity via Functional Decomposition for Better Post-Hoc Interpretability
• Nothing Else Matters: Model-Agnostic Explanations By Identifying Prediction Invariance
• Entropy-based Logic Explanations of Neural Networks
• Real-time Neural Network Inference on Extremely Weak Devices: Agile Offloading with Explainable AI
• Interpretable Deep Learning for Two-Prong Jet Classification with Jet Spectra
• Deep Learning for Computational Chemistry
• TreeView: Peeking into Deep Neural Networks Via Feature-Space Partitioning
• MAGIX: Model Agnostic Globally Interpretable Explanations
• Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking
• Tree Space Prototypes: Another Look at Making Tree Ensembles Interpretable
• An investigation on the factors affecting machine learning classifications in $\gamma$-ray astronomy
• Programs as Black-Box Explanations
• A Machine Learning Approach for Modelling Parking Duration in Urban Land-use
• Global Aggregations of Local Explanations for Black Box models
• Turbulence forecasting via Neural ODE
• Helping results assessment by adding explainable elements to the deep relevance matching model
• Counterfactual Explanations for Predictive Business Process Monitoring
• Reliable Post hoc Explanations: Modeling Uncertainty in Explainability
• Generative Causal Explanations for Graph Neural Networks
• A Biologically Interpretable Two-stage Deep Neural Network (BIT-DNN) For Vegetation Recognition From Hyperspectral Imagery
• A Detailed Study of Interpretability of Deep Neural Network based Top Taggers
• Deep Learning Assessment of galaxy morphology in S-PLUS DataRelease 1
• Individual Explanations in Machine Learning Models: A Survey for Practitioners
• Causal Interpretability for Machine Learning -- Problems, Methods and Evaluation
• Making the black-box brighter: interpreting machine learning algorithm for forecasting drilling accidents
• Learning Interpretable Models with Causal Guarantees
• Bayesian Non-parametric Quantile Process Regression and Estimation of Marginal Quantile Effects
• Deep Learning in Wide-field Surveys: Fast Analysis of Strong Lenses in Ground-based Cosmic Experiments
• Efficient computation of counterfactual explanations of LVQ models
• AdjointNet: Constraining machine learning models with physics-based codes
• Explaining the Predictions of Any Image Classifier via Decision Trees
• FlagIt: A System for Minimally Supervised Human Trafficking Indicator Mining
• Model Explainability in Deep Learning Based Natural Language Processing
• A Modified Perturbed Sampling Method for Local Interpretable Model-agnostic Explanation
• Explainable Software Defect Prediction: Are We There Yet?
• A Visual Analytics Framework for Adversarial Text Generation
• Interpreting Vision and Language Generative Models with Semantic Visual Priors
• Towards machine learning for microscopic mechanisms: a formula search for crystal structure stability based on atomic properties
• Mixture of Linear Models Co-supervised by Deep Neural Networks
• Fibres of Failure: Classifying errors in predictive processes
• GAN-based Generation and Automatic Selection of Explanations for Neural Networks
• Interpretable Machine Learning for COVID-19: An Empirical Study on Severity Prediction Task
• IFedAvg: Interpretable Data-Interoperability for Federated Learning
• Explainability in AI Based Applications: A Framework for Comparing Different Techniques
• An Extension of LIME with Improvement of Interpretability and Fidelity
• Individual Explanations in Machine Learning Models: A Case Study on Poverty Estimation
• Interpreting search result rankings through intent modeling
• Technologies for Trustworthy Machine Learning: A Survey in a Socio-Technical Context
• A Quantitative Perspective on Values of Domain Knowledge for Machine Learning
• EXS: Explainable Search Using Local Model Agnostic Interpretability
• Explaining Predictions from Tree-based Boosting Ensembles
• Connecting Interpretability and Robustness in Decision Trees through Separation
• Interpretable Visualizations with Differentiating Embedding Networks
• LEAFAGE: Example-based and Feature importance-based Explanationsfor Black-box ML models
• On Blame Attribution for Accountable Multi-Agent Sequential Decision Making
• Quantifying error contributions of computational steps, algorithms and hyperparameter choices in image classification pipelines
• Why should I not follow you? Reasons For and Reasons Against in Responsible Recommender Systems
• Location, location, location: Satellite image-based real-estate appraisal
• Reasons, Values, Stakeholders: A Philosophical Framework for Explainable Artificial Intelligence
• Data-Driven Theory-guided Learning of Partial Differential Equations using SimultaNeous Basis Function Approximation and Parameter Estimation (SNAPE)
• Model Interpretation and Explainability: Towards Creating Transparency in Prediction Models
• Discriminative Attribution from Counterfactuals
• ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes
• A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts
• Explicating feature contribution using Random Forest proximity distances
• Making Logic Learnable With Neural Networks
• Decorrelated Variable Importance
• Perturbing Inputs for Fragile Interpretations in Deep Natural Language Processing
• How to improve the interpretability of kernel learning
• A Blended Deep Learning Approach for Predicting User Intended Actions
• NeuroView: Explainable Deep Network Decision Making
• Impact of Interventional Policies Including Vaccine on Covid-19 Propagation and Socio-Economic Factors
• Causal Explanations of Image Misclassifications
• Explanation Ontology in Action: A Clinical Use-Case
• e-QRAQ: A Multi-turn Reasoning Dataset and Simulator with Explanations
• Corpus-level and Concept-based Explanations for Interpretable Document Classification
• Information-theoretic Evolution of Model Agnostic Global Explanations
• Semantic Compression for Edge-Assisted Systems
• Towards Explainable Fact Checking