Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset

Abstract

Despite the increasingly successful application of neural networks to many problems in the geosciences, their complex and nonlinear structure makes the interpretation of their predictions difficult, which limits model trust and does not allow scientists to gain physical insights about the problem at hand. Many different methods have been introduced in the emerging field of eXplainable Artificial Intelligence (XAI), which aim at attributing the network s prediction to specific features in the input domain. XAI methods are usually assessed by using benchmark datasets (like MNIST or ImageNet for image classification). However, an objective, theoretically derived ground truth for the attribution is lacking for most of these datasets, making the assessment of XAI in many cases subjective. Also, benchmark datasets specifically designed for problems in geosciences are rare. Here, we provide a framework, based on the use of additively separable functions, to generate attribution benchmark datasets for regression problems for which the ground truth of the attribution is known a priori. We generate a large benchmark dataset and train a fully connected network to learn the underlying function that was used for simulation. We then compare estimated heatmaps from different XAI methods to the ground truth in order to identify examples where specific XAI methods perform well or poorly. We believe that attribution benchmarks as the ones introduced herein are of great importance for further application of neural networks in the geosciences, and for more objective assessment and accurate implementation of XAI methods, which will increase model trust and assist in discovering new science.

This is an updated preprint version of the manuscript. This work has been published (open access) in the journal Environmental Data Science with doi: https://doi.org/10.1017/eds.2022.7. Please cite the published version. The dataset of this work is published at: https://mlhub.earth/data/csu_synthetic_attribution

References in corpus (10)

• Striving for Simplicity: The All Convolutional Net
• Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
• SmoothGrad: removing noise by adding noise
• Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
• The (Un)reliability of saliency methods
• Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience
• Indicator patterns of forced change learned by an artificial neural network
• Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey
• Towards falsifiable interpretability research
• Interpreting the Predictions of Complex ML Models by Layer-wise Relevance Propagation

Cited by in corpus (8)

• Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience
• Opening the Black-Box: A Systematic Review on Explainable AI in Remote Sensing
• Explainable Artificial Intelligence for Bayesian Neural Networks: Towards trustworthy predictions of ocean dynamics
• Non-Linear Dimensionality Reduction with a Variational Encoder Decoder to Understand Convective Processes in Climate Models
• Controlled abstention neural networks for identifying skillful predictions for regression problems
• Controlled abstention neural networks for identifying skillful predictions for classification problems
• Solving the enigma: Enhancing faithfulness and comprehensibility in explanations of deep networks
• XAI-Units: Benchmarking Explainability Methods with Unit Tests