Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models

Abstract

As deep neural networks continue to revolutionize various application domains, there is increasing interest in making these powerful models more understandable and interpretable, and narrowing down the causes of good and bad predictions. We focus on recurrent neural networks (RNNs), state of the art models in speech recognition and translation. Our approach to increasing interpretability is by combining an RNN with a hidden Markov model (HMM), a simpler and more transparent model. We explore various combinations of RNNs and HMMs: an HMM trained on LSTM states; a hybrid model where an HMM is trained first, then a small LSTM is given HMM state distributions and trained to fill in gaps in the HMM's performance; and a jointly trained hybrid model. We find that the LSTM and HMM learn complementary information about the features in the text.

presented at 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), New York, NY

References in corpus (2)

• Distilling Knowledge from Deep Networks with Applications to Healthcare Domain
• A Model Explanation System: Latest Updates and Extensions

Cited by in corpus (20)

• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• An Evaluation of the Human-Interpretability of Explanation
• How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation
• AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity
• Interpretable Recurrent Neural Networks Using Sequential Sparse Recovery
• Using Machine Learning Safely in Automotive Software: An Assessment and Adaption of Software Process Requirements in ISO 26262
• Self-Attentive Hawkes Processes
• Sequential Interpretability: Methods, Applications, and Future Direction for Understanding Deep Learning Models in the Context of Sequential Data
• Powering Hidden Markov Model by Neural Network based Generative Models
• Fibres of Failure: Classifying errors in predictive processes
• M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis
• Towards Ground Truth Explainability on Tabular Data
• Explainable Product Search with a Dynamic Relation Embedding Model
• Intelligence, physics and information -- the tradeoff between accuracy and simplicity in machine learning
• Uncertainty Estimation and Calibration with Finite-State Probabilistic RNNs
• MEME: Generating RNN Model Explanations via Model Extraction
• Understanding Recurrent Neural State Using Memory Signatures
• Maintaining The Humanity of Our Models
• Scalable Explanation of Inferences on Large Graphs
• Absolute Value Constraint: The Reason for Invalid Performance Evaluation Results of Neural Network Models for Stock Price Prediction