A Framework to Learn with Interpretation

Abstract

To tackle interpretability in deep learning, we present a novel framework to jointly learn a predictive model and its associated interpretation model. The interpreter provides both local and global interpretability about the predictive model in terms of human-understandable high level attribute functions, with minimal loss of accuracy. This is achieved by a dedicated architecture and well chosen regularization penalties. We seek for a small-size dictionary of high level attribute functions that take as inputs the outputs of selected hidden layers and whose outputs feed a linear classifier. We impose strong conciseness on the activation of attributes with an entropy-based criterion while enforcing fidelity to both inputs and outputs of the predictive model. A detailed pipeline to visualize the learnt features is also developed. Moreover, besides generating interpretable models by design, our approach can be specialized to provide post-hoc interpretations for a pre-trained neural network. We validate our approach against several state-of-the-art methods on multiple datasets and show its efficacy on both kinds of tasks.

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Towards A Rigorous Science of Interpretable Machine Learning
• Methods for Interpreting and Understanding Deep Neural Networks
• Striving for Simplicity: The All Convolutional Net
• Understanding Neural Networks Through Deep Visualization
• How to Explain Individual Classification Decisions
• This Looks Like That: Deep Learning for Interpretable Image Recognition
• A Neural Representation of Sketch Drawings
• Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
• Concept Whitening for Interpretable Image Recognition
• Concept Bottleneck Models
• Now You See Me (CME): Concept-based Model Extraction
• Explaining a black-box using Deep Variational Information Bottleneck Approach
• Learning outside the Black-Box: The pursuit of interpretable models