A copula-based visualization technique for a neural network

Abstract

Interpretability of machine learning is defined as the extent to which humans can comprehend the reason of a decision. However, a neural network is not considered interpretable due to the ambiguity in its decision-making process. Therefore, in this study, we propose a new algorithm that reveals which feature values the trained neural network considers important and which paths are mainly traced in the process of decision-making. In the proposed algorithm, the score estimated by the correlation coefficients between the neural network layers that can be calculated by applying the concept of a pair copula was defined. We compared the estimated score with the feature importance values of Random Forest, which is sometimes regarded as a highly interpretable algorithm, in the experiment and confirmed that the results were consistent with each other. This algorithm suggests an approach for compressing a neural network and its parameter tuning because the algorithm identifies the paths that contribute to the classification or prediction results.

arXiv comment8 pages, 6 figures

References in corpus (6)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Distilling the Knowledge in a Neural Network
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Do Deep Nets Really Need to be Deep?
• SmoothGrad: removing noise by adding noise
• Learning how to explain neural networks: PatternNet and PatternAttribution