On Sample Based Explanation Methods for NLP:Efficiency, Faithfulness, and Semantic Evaluation
arXiv:2106.04753
Abstract
In the recent advances of natural language processing, the scale of the state-of-the-art models and datasets is usually extensive, which challenges the application of sample-based explanation methods in many aspects, such as explanation interpretability, efficiency, and faithfulness. In this work, for the first time, we can improve the interpretability of explanations by allowing arbitrary text sequences as the explanation unit. On top of this, we implement a hessian-free method with a model faithfulness guarantee. Finally, to compare our method with the others, we propose a semantic-based evaluation metric that can better align with humans' judgment of explanations than the widely adopted diagnostic or re-training measures. The empirical results on multiple real data sets demonstrate the proposed method's superior performance to popular explanation techniques such as Influence Function or TracIn on semantic evaluation.
arXiv comment13 pages; Accepted to ACL 2021
References in corpus (13)
- Decoupled Weight Decay Regularization
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
- Axiomatic Attribution for Deep Networks
- Understanding Black-box Predictions via Influence Functions
- Understanding Neural Networks through Representation Erasure
- A Benchmark for Interpretability Methods in Deep Neural Networks
- e-SNLI: Natural Language Inference with Natural Language Explanations
- Estimating Training Data Influence by Tracing Gradient Descent
- Generative Data Augmentation for Commonsense Reasoning
- Representer Point Selection for Explaining Deep Neural Networks
- SentiHood: Targeted Aspect Based Sentiment Analysis Dataset for Urban Neighbourhoods
- Pair the Dots: Jointly Examining Training History and Test Stimuli for Model Interpretability
- Tricks from Deep Learning