Interpreting Recurrent and Attention-Based Neural Models: a Case Study on Natural Language Inference
arXiv:1808.03894
Abstract
Deep learning models have achieved remarkable success in natural language inference (NLI) tasks. While these models are widely explored, they are hard to interpret and it is often unclear how and why they actually work. In this paper, we take a step toward explaining such deep learning based models through a case study on a popular neural model for NLI. In particular, we propose to interpret the intermediate layers of NLI models by visualizing the saliency of attention and LSTM gating signals. We present several examples for which our methods are able to reveal interesting insights and identify the critical information contributing to the model decisions.
arXiv comment11 pages, 11 figures, accepted as a short paper at EMNLP 2018
References in corpus (7)
- Neural Machine Translation by Jointly Learning to Align and Translate
- Enhanced LSTM for Natural Language Inference
- Understanding Neural Networks through Representation Erasure
- A large annotated corpus for learning natural language inference
- Learned in Translation: Contextualized Word Vectors
- Natural Language Inference over Interaction Space
- Dependent Gated Reading for Cloze-Style Question Answering
Cited by in corpus (7)
- An Empirical Study of Spatial Attention Mechanisms in Deep Networks
- Explaining Explanations: Axiomatic Feature Interactions for Deep Networks
- LioNets: A Neural-Specific Local Interpretation Technique Exploiting Penultimate Layer Information
- Understanding Multi-Head Attention in Abstractive Summarization
- DNN2LR: Interpretation-inspired Feature Crossing for Real-world Tabular Data
- Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks
- SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning