Automatic Rule Extraction from Long Short Term Memory Networks

Abstract

Although deep learning models have proven effective at solving problems in natural language processing, the mechanism by which they come to their conclusions is often unclear. As a result, these models are generally treated as black boxes, yielding no insight of the underlying learned patterns. In this paper we consider Long Short Term Memory networks (LSTMs) and demonstrate a new approach for tracking the importance of a given input to the LSTM for a given output. By identifying consistently important patterns of words, we are able to distill state of the art LSTMs on sentiment analysis and question answering into a set of representative phrases. This representation is then quantitatively validated by using the extracted phrases to construct a simple, rule-based classifier which approximates the output of the LSTM.

ICLR 2017 accepted paper

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Convolutional Neural Networks for Sentence Classification
• Exploring the Limits of Language Modeling
• Large-scale Simple Question Answering with Memory Networks
• Ask the GRU: Multi-Task Learning for Deep Text Recommendations
• Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering

Cited by in corpus (19)

• Interpretable machine learning: definitions, methods, and applications
• Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs
• TimeSHAP: Explaining Recurrent Models through Sequence Perturbations
• Hierarchical interpretations for neural network predictions
• Explainable Artificial Intelligence: a Systematic Review
• Learning Memory Access Patterns
• Towards Binary-Valued Gates for Robust LSTM Training
• Interpretations are useful: penalizing explanations to align neural networks with prior knowledge
• Understanding Hidden Memories of Recurrent Neural Networks
• Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
• Interpretable Deep Learning under Fire
• Sequential Interpretability: Methods, Applications, and Future Direction for Understanding Deep Learning Models in the Context of Sequential Data
• A Comparative Study of Rule Extraction for Recurrent Neural Networks
• Evaluating neural network explanation methods using hybrid documents and morphological agreement
• Distance and Equivalence between Finite State Machines and Recurrent Neural Networks: Computational results
• An Empirical Evaluation of Rule Extraction from Recurrent Neural Networks
• Understanding Recurrent Neural State Using Memory Signatures
• Persistence pays off: Paying Attention to What the LSTM Gating Mechanism Persists
• Explaining the Deep Natural Language Processing by Mining Textual Interpretable Features