TreeView: Peeking into Deep Neural Networks Via Feature-Space Partitioning

Abstract

With the advent of highly predictive but opaque deep learning models, it has become more important than ever to understand and explain the predictions of such models. Existing approaches define interpretability as the inverse of complexity and achieve interpretability at the cost of accuracy. This introduces a risk of producing interpretable but misleading explanations. As humans, we are prone to engage in this kind of behavior \cite{mythos}. In this paper, we take a step in the direction of tackling the problem of interpretability without compromising the model accuracy. We propose to build a Treeview representation of the complex model via hierarchical partitioning of the feature space, which reveals the iterative rejection of unlikely class labels until the correct association is predicted.

arXiv commentPresented at NIPS 2016 Workshop on Interpretable Machine Learning in Complex Systems

References in corpus (2)

• Model-Agnostic Interpretability of Machine Learning
• Methods and Models for Interpretable Linear Classification

Cited by in corpus (8)

• Explainable Artificial Intelligence: a Systematic Review
• Testing and verification of neural-network-based safety-critical control software: A systematic literature review
• Generative Counterfactual Introspection for Explainable Deep Learning
• Contrastive Explanations with Local Foil Trees
• Why model why? Assessing the strengths and limitations of LIME
• Fibres of Failure: Classifying errors in predictive processes
• Embedded Encoder-Decoder in Convolutional Networks Towards Explainable AI
• Unsupervised discovery of Interpretable Visual Concepts