Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges

Abstract

We present a brief history of the field of interpretable machine learning (IML), give an overview of state-of-the-art interpretation methods, and discuss challenges. Research in IML has boomed in recent years. As young as the field is, it has over 200 years old roots in regression modeling and rule-based machine learning, starting in the 1960s. Recently, many new IML methods have been proposed, many of them model-agnostic, but also interpretation techniques specific to deep learning and tree-based ensembles. IML methods either directly analyze model components, study sensitivity to input perturbations, or analyze local or global surrogate approximations of the ML model. The field approaches a state of readiness and stability, with many methods not only proposed in research, but also implemented in open-source software. But many important challenges remain for IML, such as dealing with dependent features, causal interpretation, and uncertainty estimation, which need to be resolved for its successful application to scientific problems. A further challenge is a missing rigorous definition of interpretability, which is accepted by the community. To address the challenges and advance the field, we urge to recall our roots of interpretable, data-driven modeling in statistics and (rule-based) ML, but also to consider other areas such as sensitivity analysis, causal inference, and the social sciences.

References in corpus (46)

• Deep Learning in Neural Networks: An Overview
• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• Interpretable machine learning: definitions, methods, and applications
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Understanding Black-box Predictions via Influence Functions
• Predictive learning via rule ensembles
• Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations
• All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• Sanity Checks for Saliency Maps
• Towards Explainable Artificial Intelligence
• Consistent Individualized Feature Attribution for Tree Ensembles
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• The What-If Tool: Interactive Probing of Machine Learning Models
• Actionable Recourse in Linear Classification
• Variable importance in binary regression trees and forests
• Supersparse Linear Integer Models for Optimized Medical Scoring Systems
• InterpretML: A Unified Framework for Machine Learning Interpretability
• Distilling a Neural Network Into a Soft Decision Tree
• Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models
• Metrics for Explainable AI: Challenges and Prospects
• Multi-Objective Counterfactual Explanations
• Feature relevance quantification in explainable AI: A causal problem
• Fast calibrated additive quantile regression
• Interpreting Blackbox Models via Model Extraction
• Visualizing the Feature Importance for Black Box Models
• Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking
• Causality for Machine Learning
• Model-agnostic Feature Importance and Effects with Dependent Features -- A Conditional Subgroup Approach
• Explainable Artificial Intelligence: a Systematic Review
• A Simple and Effective Model-Based Variable Importance Measure
• Quantifying Model Complexity via Functional Decomposition for Better Post-Hoc Interpretability
• Assessing the Local Interpretability of Machine Learning Models
• A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning
• Stakeholders in Explainable AI
• Locally Interpretable Models and Effects based on Supervised Partitioning (LIME-SUP)
• MAGIX: Model Agnostic Globally Interpretable Explanations
• PS^2-Net: A Locally and Globally Aware Network for Point-Based Semantic Segmentation
• Relative Feature Importance
• OptiLIME: Optimized LIME Explanations for Diagnostic Computer Algorithms
• Efficient nonparametric statistical inference on population feature importance using Shapley values
• TIP: Typifying the Interpretability of Procedures
• A study of data and label shift in the LIME framework
• audioLIME: Listenable Explanations Using Source Separation
• MeLIME: Meaningful Local Explanation for Machine Learning Models

Cited by in corpus (13)

• Choice modelling in the age of machine learning -- discussion paper
• A Prescriptive Learning Analytics Framework: Beyond Predictive Modelling and onto Explainable AI with Prescriptive Analytics and ChatGPT
• Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions
• A Survey on Neural Network Interpretability
• Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena
• Explainable Ensemble-Based Machine Learning Models for Detecting the Presence of Cirrhosis in Hepatitis C Patients
• Transforming Feature Space to Interpret Machine Learning Models
• Explaining classifiers to understand coarse-grained models
• Explanations as Programs in Probabilistic Logic Programming
• BDD4BNN: A BDD-based Quantitative Analysis Framework for Binarized Neural Networks
• Neural-to-Tree Policy Distillation with Policy Improvement Criterion
• Interpretable machine-learning identification of the crossover from subradiance to superradiance in an atomic array
• A Feature Selection Layer Approach for Multi-Class Classification on Neural Networks