Regularizing Black-box Models for Improved Interpretability

Abstract

Most of the work on interpretable machine learning has focused on designing either inherently interpretable models, which typically trade-off accuracy for interpretability, or post-hoc explanation systems, whose explanation quality can be unpredictable. Our method, ExpO, is a hybridization of these approaches that regularizes a model for explanation quality at training time. Importantly, these regularizers are differentiable, model agnostic, and require no domain knowledge to define. We demonstrate that post-hoc explanations for ExpO-regularized models have better explanation quality, as measured by the common fidelity and stability metrics. We verify that improving these metrics leads to significantly more useful explanations with a user study on a realistic task.

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• SmoothGrad: removing noise by adding noise
• Sanity Checks for Saliency Maps
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• Towards Robust Interpretability with Self-Explaining Neural Networks
• Adversarial Robustness through Local Linearization
• Model Agnostic Supervised Local Explanations
• Beyond Sparsity: Tree Regularization of Deep Models for Interpretability
• On the Robustness of Interpretability Methods
• Interpretation of Neural Networks is Fragile
• Contextual Explanation Networks
• Regional Tree Regularization for Interpretability in Black Box Models
• Game-Theoretic Interpretability for Temporal Modeling
• Functional Transparency for Structured Data: a Game-Theoretic Approach

Cited by in corpus (10)

• Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations
• Quantifying Model Complexity via Functional Decomposition for Better Post-Hoc Interpretability
• A Survey on Neural Network Interpretability
• Interpretable Machine Learning: Moving From Mythos to Diagnostics
• FINER: Enhancing State-of-the-art Classifiers with Feature Attribution to Facilitate Security Analysis
• Regularizing Reasons for Outfit Evaluation with Gradient Penalty
• Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers
• NLS: an accurate and yet easy-to-interpret regression method
• Shapley Explanation Networks
• Learning to Predict with Supporting Evidence: Applications to Clinical Risk Prediction