Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Models

Abstract

The impressive performance of neural networks on natural language processing tasks attributes to their ability to model complicated word and phrase compositions. To explain how the model handles semantic compositions, we study hierarchical explanation of neural network predictions. We identify non-additivity and context independent importance attributions within hierarchies as two desirable properties for highlighting word and phrase compositions. We show some prior efforts on hierarchical explanations, e.g. contextual decomposition, do not satisfy the desired properties mathematically, leading to inconsistent explanation quality in different models. In this paper, we start by proposing a formal and general way to quantify the importance of each word and phrase. Following the formulation, we propose Sampling and Contextual Decomposition (SCD) algorithm and Sampling and Occlusion (SOC) algorithm. Human and metrics evaluation on both LSTM models and BERT Transformer models on multiple datasets show that our algorithms outperform prior hierarchical explanation algorithms. Our algorithms help to visualize semantic composition captured by models, extract classification rules and improve human trust of models. Project page: https://inklab.usc.edu/hiexpl/

ICLR 2020

References in corpus (10)

• Axiomatic Attribution for Deep Networks
• Learning Important Features Through Propagating Activation Differences
• Understanding Neural Networks through Representation Erasure
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs
• Hierarchical interpretations for neural network predictions
• Interpretation of Prediction Models Using the Input Gradient
• BERT Rediscovers the Classical NLP Pipeline
• Detecting Statistical Interactions from Neural Network Weights
• Evaluating neural network explanation methods using hybrid documents and morphological agreement

Cited by in corpus (11)

• How does this interaction affect me? Interpretable attribution for feature interactions
• Towards generalisable hate speech detection: a review on obstacles and solutions
• Refining Language Models with Compositional Explanations
• Generating Hierarchical Explanations on Text Classification via Feature Interaction Detection
• ProtoryNet - Interpretable Text Classification Via Prototype Trajectories
• Technical Note: Game-Theoretic Interactions of Different Orders
• Interpreting and Boosting Dropout from a Game-Theoretic View
• Interpretation of NLP models through input marginalization
• Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks
• Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis
• Interpretable Artificial Intelligence through the Lens of Feature Interaction