Simplifying the explanation of deep neural networks with sufficient and necessary feature-sets: case of text classification
arXiv:2010.03724
Abstract
During the last decade, deep neural networks (DNN) have demonstrated impressive performances solving a wide range of problems in various domains such as medicine, finance, law, etc. Despite their great performances, they have long been considered as black-box systems, providing good results without being able to explain them. However, the inability to explain a system decision presents a serious risk in critical domains such as medicine where people's lives are at stake. Several works have been done to uncover the inner reasoning of deep neural networks. Saliency methods explain model decisions by assigning weights to input features that reflect their contribution to the classifier decision. However, not all features are necessary to explain a model decision. In practice, classifiers might strongly rely on a subset of features that might be sufficient to explain a particular decision. The aim of this article is to propose a method to simplify the prediction explanation of One-Dimensional (1D) Convolutional Neural Networks (CNN) by identifying sufficient and necessary features-sets. We also propose an adaptation of Layer-wise Relevance Propagation for 1D-CNN. Experiments carried out on multiple datasets show that the distribution of relevance among features is similar to that obtained with a well known state of the art model. Moreover, the sufficient and necessary features extracted perceptually appear convincing to humans.
arXiv commentFigure 3.d has been replaced : the distribution of relevances was shifted by mistake, Fig. 4 has been replaced : Sufficient features was wrongly highlighted, Fig. 2 has been replaced : The sum z was incorrect. Related calculations has been updated accordingly. Notes : These changes are minor changes and do not alter the comprehension of the findings highlted in the work
References in corpus (8)
- A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
- Visualizing and Understanding Recurrent Networks
- Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
- Opening the Black Box of Deep Neural Networks via Information
- Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
- Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
- Learning how to explain neural networks: PatternNet and PatternAttribution
- "What is Relevant in a Text Document?": An Interpretable Machine Learning Approach