Physically Interpretable Neural Networks for the Geosciences: Applications to Earth System Variability

Abstract

Neural networks have become increasingly prevalent within the geosciences, although a common limitation of their usage has been a lack of methods to interpret what the networks learn and how they make decisions. As such, neural networks have often been used within the geosciences to most accurately identify a desired output given a set of inputs, with the interpretation of what the network learns used as a secondary metric to ensure the network is making the right decision for the right reason. Neural network interpretation techniques have become more advanced in recent years, however, and we therefore propose that the ultimate objective of using a neural network can also be the interpretation of what the network has learned rather than the output itself. We show that the interpretation of neural networks can enable the discovery of scientifically meaningful connections within geoscientific data. In particular, we use two methods for neural network interpretation called backwards optimization and layerwise relevance propagation, both of which project the decision pathways of a network back onto the original input dimensions. To the best of our knowledge, LRP has not yet been applied to geoscientific research, and we believe it has great potential in this area. We show how these interpretation techniques can be used to reliably infer scientifically meaningful information from neural networks by applying them to common climate patterns. These results suggest that combining interpretable neural networks with novel scientific hypotheses will open the door to many new avenues in neural network-related geoscience research.

arXiv commentThe second version of this manuscript is currently under review at the Journal of Advances in Modeling Earth Systems (JAMES)

References in corpus (9)

• An overview of gradient descent optimization algorithms
• Methods for Interpreting and Understanding Deep Neural Networks
• Understanding Neural Networks Through Deep Visualization
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Deep learning to represent sub-grid processes in climate models
• Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• Spatially Extended Tests of a Neural Network Parametrization Trained by Coarse-graining
• Explaining and Interpreting LSTMs

Cited by in corpus (23)

• WeatherBench: A benchmark dataset for data-driven weather forecasting
• Differentiable modeling to unify machine learning and physical models and advance Geosciences
• Will Artificial Intelligence supersede Earth System and Climate Models?
• Weak SINDy For Partial Differential Equations
• Bridging observation, theory and numerical simulation of the ocean using Machine Learning
• Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models
• Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset
• Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience
• Indicator patterns of forced change learned by an artificial neural network
• Explainable Artificial Intelligence for Bayesian Neural Networks: Towards trustworthy predictions of ocean dynamics
• Assessing the Potential of Deep Learning for Emulating Cloud Superparameterization in Climate Models with Real-Geography Boundary Conditions
• Time Series Predictions in Unmonitored Sites: A Survey of Machine Learning Techniques in Water Resources
• When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System
• Machine learning applications for weather and climate need greater focus on extremes
• Controlled abstention neural networks for identifying skillful predictions for classification problems
• Identifying Opportunities for Skillful Weather Prediction with Interpretable Neural Networks
• Insightful analysis of historical sources at scales beyond human capabilities using unsupervised Machine Learning and XAI
• Physically Explainable Deep Learning for Convective Initiation Nowcasting Using GOES-16 Satellite Observations
• Loosely Conditioned Emulation of Global Climate Models With Generative Adversarial Networks
• Satellite galaxies' drag on field stars in the Milky Way
• Evaluation, Tuning and Interpretation of Neural Networks for Meteorological Applications
• Towards Physically Consistent Deep Learning For Climate Model Parameterizations
• Example-Based Concept Analysis Framework for Deep Weather Forecast Models