Explainable Artificial Intelligence for Bayesian Neural Networks: Towards trustworthy predictions of ocean dynamics

Abstract

The trustworthiness of neural networks is often challenged because they lack the ability to express uncertainty and explain their skill. This can be problematic given the increasing use of neural networks in high stakes decision-making such as in climate change applications. We address both issues by successfully implementing a Bayesian Neural Network (BNN), where parameters are distributions rather than deterministic, and applying novel implementations of explainable AI (XAI) techniques. The uncertainty analysis from the BNN provides a comprehensive overview of the prediction more suited to practitioners' needs than predictions from a classical neural network. Using a BNN means we can calculate the entropy (i.e. uncertainty) of the predictions and determine if the probability of an outcome is statistically significant. To enhance trustworthiness, we also spatially apply the two XAI techniques of Layer-wise Relevance Propagation (LRP) and SHapley Additive exPlanation (SHAP) values. These XAI methods reveal the extent to which the BNN is suitable and/or trustworthy. Using two techniques gives a more holistic view of BNN skill and its uncertainty, as LRP considers neural network parameters, whereas SHAP considers changes to outputs. We verify these techniques using comparison with intuition from physical theory. The differences in explanation identify potential areas where new physical theory guided studies are needed.

arXiv comment25 pages, 11 figures

References in corpus (24)

• Adam: A Method for Stochastic Optimization
• A Unified Approach to Interpreting Model Predictions
• On Calibration of Modern Neural Networks
• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users
• Physically Interpretable Neural Networks for the Geosciences: Applications to Earth System Variability
• Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification
• "What is Relevant in a Text Document?": An Interpretable Machine Learning Approach
• TensorFlow Distributions
• Statistical Postprocessing for Weather Forecasts -- Review, Challenges and Avenues in a Big Data World
• Practical Deep Learning with Bayesian Principles
• Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset
• Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience
• Combining distribution-based neural networks to predict weather forecast probabilities
• The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective
• Rethinking Explainability as a Dialogue: A Practitioner's Perspective
• Trustworthy AI: From Principles to Practices
• On the Validity of Bayesian Neural Networks for Uncertainty Estimation
• Reliable Post hoc Explanations: Modeling Uncertainty in Explainability
• How Much Can I Trust You? -- Quantifying Uncertainties in Explaining Neural Networks
• Learning Global Pairwise Interactions with Bayesian Neural Networks
• Being Bayesian about Categorical Probability
• Prior choice affects ability of Bayesian neural networks to identify unknowns
• Objective discovery of dominant dynamical processes with intelligible machine learning

Cited by in corpus (2)

• Learning Closed-form Equations for Subgrid-scale Closures from High-fidelity Data: Promises and Challenges
• Uncertainties of Satellite-based Essential Climate Variables from Deep Learning