Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers
arXiv:2010.00667
Abstract
To build an interpretable neural text classifier, most of the prior work has focused on designing inherently interpretable models or finding faithful explanations. A new line of work on improving model interpretability has just started, and many existing methods require either prior information or human annotations as additional inputs in training. To address this limitation, we propose the variational word mask (VMASK) method to automatically learn task-specific important words and reduce irrelevant information on classification, which ultimately improves the interpretability of model predictions. The proposed method is evaluated with three neural text classifiers (CNN, LSTM, and BERT) on seven benchmark text classification datasets. Experiments show the effectiveness of VMASK in improving both model prediction accuracy and interpretability.
arXiv commentEMNLP 2020
References in corpus (14)
- Distributed Representations of Words and Phrases and their Compositionality
- Categorical Reparameterization with Gumbel-Softmax
- Axiomatic Attribution for Deep Networks
- The information bottleneck method
- Variational Inference with Normalizing Flows
- Recurrent Neural Network for Text Classification with Multi-Task Learning
- A C-LSTM Neural Network for Text Classification
- Towards Robust Interpretability with Self-Explaining Neural Networks
- Deep Variational Information Bottleneck
- Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
- e-SNLI: Natural Language Inference with Natural Language Explanations
- Restricting the Flow: Information Bottlenecks for Attribution
- Regularizing Black-box Models for Improved Interpretability
- A Tutorial on Deep Latent Variable Models of Natural Language