How Much Can I Trust You? -- Quantifying Uncertainties in Explaining Neural Networks

Abstract

Explainable AI (XAI) aims to provide interpretations for predictions made by learning machines, such as deep neural networks, in order to make the machines more transparent for the user and furthermore trustworthy also for applications in e.g. safety-critical areas. So far, however, no methods for quantifying uncertainties of explanations have been conceived, which is problematic in domains where a high confidence in explanations is a prerequisite. We therefore contribute by proposing a new framework that allows to convert any arbitrary explanation method for neural networks into an explanation method for Bayesian neural networks, with an in-built modeling of uncertainties. Within the Bayesian framework a network's weights follow a distribution that extends standard single explanation scores and heatmaps to distributions thereof, in this manner translating the intrinsic network model uncertainties into a quantification of explanation uncertainties. This allows us for the first time to carve out uncertainties associated with a model explanation and subsequently gauge the appropriate level of explanation confidence for a user (using percentiles). We demonstrate the effectiveness and usefulness of our approach extensively in various experiments, both qualitatively and quantitatively.

arXiv comment12 pages, 10 figures

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Methods for Interpreting and Understanding Deep Neural Networks
• Understanding Neural Networks Through Deep Visualization
• How to Explain Individual Classification Decisions
• Variational Dropout and the Local Reparameterization Trick
• Variational Dropout Sparsifies Deep Neural Networks
• From Clustering to Cluster Explanations via Neural Networks
• Towards Explaining Anomalies: A Deep Taylor Decomposition of One-Class Models
• How Good is the Bayes Posterior in Deep Neural Networks Really?
• Feature Importance Measure for Non-linear Learning Algorithms
• Towards computational fluorescence microscopy: Machine learning-based integrated prediction of morphological and molecular tumor profiles

Cited by in corpus (2)

• Explainable Artificial Intelligence for Bayesian Neural Networks: Towards trustworthy predictions of ocean dynamics
• On the use of uncertainty in classifying Aedes Albopictus mosquitoes