MEME: Generating RNN Model Explanations via Model Extraction

Abstract

Recurrent Neural Networks (RNNs) have achieved remarkable performance on a range of tasks. A key step to further empowering RNN-based approaches is improving their explainability and interpretability. In this work we present MEME: a model extraction approach capable of approximating RNNs with interpretable models represented by human-understandable concepts and their interactions. We demonstrate how MEME can be applied to two multivariate, continuous data case studies: Room Occupation Prediction, and In-Hospital Mortality Prediction. Using these case-studies, we show how our extracted models can be used to interpret RNNs both locally and globally, by approximating RNN decision-making via interpretable concept interactions.

arXiv commentPresented at the HAMLETS workshop at the 34th Conference on Neural Information Processing Systems (NeurIPS 2020)

References in corpus (11)

• Scikit-learn: Machine Learning in Python
• Deep learning for time series classification: a review
• Sanity Checks for Saliency Maps
• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
• Interpreting Blackbox Models via Model Extraction
• Beyond Sparsity: Tree Regularization of Deep Models for Interpretability
• Explaining Classifiers with Causal Concept Effect (CaCE)
• Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models
• Learning Deterministic Weighted Automata with Queries and Counterexamples
• Discovering Subdimensional Motifs of Different Lengths in Large-Scale Multivariate Time Series
• Efficient Algorithms for Generating Provably Near-Optimal Cluster Descriptors for Explainability