Programs as Black-Box Explanations

Abstract

Recent work in model-agnostic explanations of black-box machine learning has demonstrated that interpretability of complex models does not have to come at the cost of accuracy or model flexibility. However, it is not clear what kind of explanations, such as linear models, decision trees, and rule lists, are the appropriate family to consider, and different tasks and models may benefit from different kinds of explanations. Instead of picking a single family of representations, in this work we propose to use "programs" as model-agnostic explanations. We show that small programs can be expressive yet intuitive as explanations, and generalize over a number of existing interpretable families. We propose a prototype program induction method based on simulated annealing that approximates the local behavior of black-box classifiers around a specific prediction using random perturbations. Finally, we present preliminary application on small datasets and show that the generated explanations are intuitive and accurate for a number of classifiers.

Presented at NIPS 2016 Workshop on Interpretable Machine Learning in Complex Systems

References in corpus (6)

• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• Model-Agnostic Interpretability of Machine Learning
• Church: a language for generative models
• Supersparse Linear Integer Models for Optimized Medical Scoring Systems
• Programming with a Differentiable Forth Interpreter
• TerpreT: A Probabilistic Programming Language for Program Induction

Cited by in corpus (7)

• Local Rule-Based Explanations of Black Box Decision Systems
• Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects
• An Evaluation of the Human-Interpretability of Explanation
• How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation
• Regional Tree Regularization for Interpretability in Black Box Models
• Training Machine Learning Models by Regularizing their Explanations
• Identifying the Most Explainable Classifier