New Perspective of Interpretability of Deep Neural Networks
arXiv:1909.07156
Abstract
Deep neural networks (DNNs) are known as black-box models. In other words, it is difficult to interpret the internal state of the model. Improving the interpretability of DNNs is one of the hot research topics. However, at present, the definition of interpretability for DNNs is vague, and the question of what is a highly explanatory model is still controversial. To address this issue, we provide the definition of the human predictability of the model, as a part of the interpretability of the DNNs. The human predictability proposed in this paper is defined by easiness to predict the change of the inference when perturbating the model of the DNNs. In addition, we introduce one example of high human-predictable DNNs. We discuss that our definition will help to the research of the interpretability of the DNNs considering various types of applications.
References in corpus (7)
- Neural Architecture Search with Reinforcement Learning
- Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
- Neural Architecture Search: A Survey
- All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
- Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
- How to Explain Individual Classification Decisions
- Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers