Beyond Sparsity: Tree Regularization of Deep Models for Interpretability

Abstract

The lack of interpretability remains a key barrier to the adoption of deep models in many applications. In this work, we explicitly regularize deep models so human users might step through the process behind their predictions in little time. Specifically, we train deep time-series models so their class-probability predictions have high accuracy while being closely modeled by decision trees with few nodes. Using intuitive toy examples as well as medical tasks for treating sepsis and HIV, we demonstrate that this new tree regularization yields models that are easier for humans to simulate than simpler L1 or L2 penalties without sacrificing predictive power.

To appear in AAAI 2018. Contains 9-page main paper and appendix with supplementary material

Cited by in corpus (37)

• A Survey on the Explainability of Supervised Machine Learning
• The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies
• What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
• Explainable Matrix -- Visualization for Global and Local Interpretability of Random Forest Classification Ensembles
• Reinforcement Learning in Healthcare: A Survey
• DeepAID: Interpreting and Improving Deep Learning-based Anomaly Detection in Security Applications
• Explainable Artificial Intelligence: a Systematic Review
• Benchmarking Deep Learning Interpretability in Time Series Predictions
• Explainable Diabetic Retinopathy Detection and Retinal Image Generation
• A Survey of Explainable Artificial Intelligence (XAI) in Financial Time Series Forecasting
• A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning
• On Interpretability of Artificial Neural Networks: A Survey
• Explaining Explanations: Axiomatic Feature Interactions for Deep Networks
• Regularizing Black-box Models for Improved Interpretability
• A Survey on Neural Network Interpretability
• Medical Time Series Classification with Hierarchical Attention-based Temporal Convolutional Networks: A Case Study of Myotonic Dystrophy Diagnosis
• Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond
• Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solution
• Exathlon: A Benchmark for Explainable Anomaly Detection over Time Series
• Personalized explanation in machine learning: A conceptualization
• Generative Grading: Near Human-level Accuracy for Automated Feedback on Richly Structured Problems
• Regional Tree Regularization for Interpretability in Black Box Models
• Learning with Interpretable Structure from Gated RNN
• Interpretable Learning-to-Rank with Generalized Additive Models
• Defining and Quantifying the Emergence of Sparse Concepts in DNNs
• Graph Neural Networks Including Sparse Interpretability
• Interpretable Time-series Classification on Few-shot Samples
• Show or Suppress? Managing Input Uncertainty in Machine Learning Model Explanations
• Knowledge Representing: Efficient, Sparse Representation of Prior Knowledge for Knowledge Distillation
• Fuzzy Logic Interpretation of Quadratic Networks
• MEME: Generating RNN Model Explanations via Model Extraction
• Automated Testing of AI Models
• Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees
• Training Machine Learning Models by Regularizing their Explanations
• Explainable Deep Modeling of Tabular Data using TableGraphNet
• Semantically Rich Local Dataset Generation for Explainable AI in Genomics
• Exact and Consistent Interpretation of Piecewise Linear Models Hidden behind APIs: A Closed Form Solution