Interpreting Blackbox Models via Model Extraction

Abstract

Interpretability has become incredibly important as machine learning is increasingly used to inform consequential decisions. We propose to construct global explanations of complex, blackbox models in the form of a decision tree approximating the original model---as long as the decision tree is a good approximation, then it mirrors the computation performed by the blackbox model. We devise a novel algorithm for extracting decision tree explanations that actively samples new training points to avoid overfitting. We evaluate our algorithm on a random forest to predict diabetes risk and a learned controller for cart-pole. Compared to several baselines, our decision trees are both substantially more accurate and equally or more interpretable based on a user study. Finally, we describe several insights provided by our interpretations, including a causal issue validated by a physician.

References in corpus (8)

• Towards A Rigorous Science of Interpretable Machine Learning
• Learning Important Features Through Propagating Activation Differences
• Equality of Opportunity in Supervised Learning
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• Supersparse Linear Integer Models for Optimized Medical Scoring Systems
• Distilling a Neural Network Into a Soft Decision Tree
• Interpreting Tree Ensembles with inTrees
• GENESIM: genetic extraction of a single, interpretable model

Cited by in corpus (41)

• A Survey on the Explainability of Supervised Machine Learning
• The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies
• Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges
• Deep Weighted Averaging Classifiers
• MAGIX: Model Agnostic Globally Interpretable Explanations
• Considerations When Learning Additive Explanations for Black-Box Models
• Model Distillation for Revenue Optimization: Interpretable Personalized Pricing
• On the Art and Science of Machine Learning Explanations
• The Price of Interpretability
• Born-Again Tree Ensembles
• Trepan Reloaded: A Knowledge-driven Approach to Explaining Artificial Neural Networks
• TIP: Typifying the Interpretability of Procedures
• Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solution
• TED: Teaching AI to Explain its Decisions
• Proposed Guidelines for the Responsible Use of Explainable Machine Learning
• Explaining Any ML Model? -- On Goals and Capabilities of XAI
• Learning Interpretable Models with Causal Guarantees
• CDT: Cascading Decision Trees for Explainable Reinforcement Learning
• Teaching Meaningful Explanations
• Learning Global Transparent Models Consistent with Local Contrastive Explanations
• Rationalizing Predictions by Adversarial Information Calibration
• "Why Should I Trust Interactive Learners?" Explaining Interactive Queries of Classifiers to Users
• Dynamic Batch Learning in High-Dimensional Sparse Linear Contextual Bandits
• Machine Guides, Human Supervises: Interactive Learning with Global Explanations
• Evaluating Explanation Methods for Neural Machine Translation
• Human Understandable Explanation Extraction for Black-box Classification Models Based on Matrix Factorization
• Optimal Explanations of Linear Models
• Efficient Encrypted Inference on Ensembles of Decision Trees
• Optimal Piecewise Local-Linear Approximations
• Teaching AI to Explain its Decisions Using Embeddings and Multi-Task Learning
• Fuzzy Logic Interpretation of Quadratic Networks
• Interpreting Deep Learning Model Using Rule-based Method
• MEME: Generating RNN Model Explanations via Model Extraction
• Bandits for Learning to Explain from Explanations
• Toward Machine-Guided, Human-Initiated Explanatory Interactive Learning
• An Active Approach for Model Interpretation
• Generalized Adversarial Distances to Efficiently Discover Classifier Errors
• Information-theoretic Evolution of Model Agnostic Global Explanations
• SONG: Self-Organizing Neural Graphs
• Learning Models for Actionable Recourse
• AURORA: Auditing PageRank on Large Graphs