Distilling a Neural Network Into a Soft Decision Tree
arXiv:1711.09784
Abstract
Deep neural networks have proved to be a very effective way to perform classification tasks. They excel when the input data is high dimensional, the relationship between the input and the output is complicated, and the number of labeled training examples is large. But it is hard to explain why a learned network makes a particular classification decision on a particular test case. This is due to their reliance on distributed hierarchical representations. If we could take the knowledge acquired by the neural net and express the same knowledge in a model that relies on hierarchical decisions instead, explaining a particular decision would be much easier. We describe a way of using a trained neural net to create a type of soft decision tree that generalizes better than one learned directly from the training data.
presented at the CEX workshop at AI*IA 2017 conference
References in corpus (1)
Cited by in corpus (48)
- A Survey on the Explainability of Supervised Machine Learning
- Benchmarking Deep Learning Interpretability in Time Series Predictions
- Neural Network Attributions: A Causal Perspective
- Explainable Machine Learning with Prior Knowledge: An Overview
- FEED: Feature-level Ensemble for Knowledge Distillation
- What does it mean to understand a neural network?
- Scalable Rule-Based Representation Learning for Interpretable Classification
- Global Explanations of Neural Networks: Mapping the Landscape of Predictions
- Interpret Federated Learning with Shapley Values
- Semantics and explanation: why counterfactual explanations produce adversarial examples in deep neural networks
- Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond
- Transferring Inductive Biases through Knowledge Distillation
- Improving the Interpretability of Deep Neural Networks with Knowledge Distillation
- On Relating 'Why?' and 'Why Not?' Explanations
- Hybrid Predictive Model: When an Interpretable Model Collaborates with a Black-box Model
- CDT: Cascading Decision Trees for Explainable Reinforcement Learning
- Explaining Deep Neural Networks using Unsupervised Clustering
- Soft Gradient Boosting Machine
- A novel method for extracting interpretable knowledge from a spiking neural classifier with time-varying synaptic weights
- Deep CNNs for Peripheral Blood Cell Classification
- Incorporating Priors with Feature Attribution on Text Classification
- A New Approach for Explainable Multiple Organ Annotation with Few Data
- Explainable Multivariate Time Series Classification: A Deep Neural Network Which Learns To Attend To Important Variables As Well As Informative Time Intervals
- Explaining Neural Networks Semantically and Quantitatively
- Adaptive wavelet distillation from neural networks through interpretations
- Synthesising Reinforcement Learning Policies through Set-Valued Inductive Rule Learning
- Coloring the Black Box: Visualizing neural network behavior with a self-introspective model
- A Causal Lens for Peeking into Black Box Predictive Models: Predictive Model Interpretation via Causal Attribution
- Rectified Decision Trees: Exploring the Landscape of Interpretable and Effective Machine Learning
- A Survey of Techniques All Classifiers Can Learn from Deep Networks: Models, Optimizations, and Regularization
- Deep Transfer Learning with Ridge Regression
- Efficient Encrypted Inference on Ensembles of Decision Trees
- Formal Methods with a Touch of Magic
- Interpretable Time-series Classification on Few-shot Samples
- Transparent Classification with Multilayer Logical Perceptrons and Random Binarization
- Extracting Optimal Explanations for Ensemble Trees via Logical Reasoning
- Neural-to-Tree Policy Distillation with Policy Improvement Criterion
- BDD4BNN: A BDD-based Quantitative Analysis Framework for Binarized Neural Networks
- Extracting Interpretable Concept-Based Decision Trees from CNNs
- Channel Pruning via Multi-Criteria based on Weight Dependency
- Computing Class Hierarchies from Classifiers
- Games for Fairness and Interpretability
- Back to Square One: Superhuman Performance in Chutes and Ladders Through Deep Neural Networks and Tree Search
- Adaptive Bayesian Reticulum
- Computational principles of intelligence: learning and reasoning with neural networks
- Efficient Modelling Across Time of Human Actions and Interactions
- Learning Multi-Layered GBDT Via Back Propagation
- Making CNNs Interpretable by Building Dynamic Sequential Decision Forests with Top-down Hierarchy Learning