Differentiable modeling to unify machine learning and physical models and advance Geosciences

Abstract

Process-Based Modeling (PBM) and Machine Learning (ML) are often perceived as distinct paradigms in the geosciences. Here we present differentiable geoscientific modeling as a powerful pathway toward dissolving the perceived barrier between them and ushering in a paradigm shift. For decades, PBM offered benefits in interpretability and physical consistency but struggled to efficiently leverage large datasets. ML methods, especially deep networks, presented strong predictive skills yet lacked the ability to answer specific scientific questions. While various methods have been proposed for ML-physics integration, an important underlying theme -- differentiable modeling -- is not sufficiently recognized. Here we outline the concepts, applicability, and significance of differentiable geoscientific modeling (DG). "Differentiable" refers to accurately and efficiently calculating gradients with respect to model variables, critically enabling the learning of high-dimensional unknown relationships. DG refers to a range of methods connecting varying amounts of prior knowledge to neural networks and training them together, capturing a different scope than physics-guided machine learning and emphasizing first principles. Preliminary evidence suggests DG offers better interpretability and causality than ML, improved generalizability and extrapolation capability, and strong potential for knowledge discovery, while approaching the performance of purely data-driven ML. DG models require less training data while scaling favorably in performance and efficiency with increasing amounts of data. With DG, geoscientists may be better able to frame and investigate questions, test hypotheses, and discover unrecognized linkages.

References in corpus (12)

• Methods for Interpreting and Understanding Deep Neural Networks
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• Crop Yield Prediction Using Deep Neural Networks
• Deep Learning of Subsurface Flow via Theory-guided Neural Network
• Differentiable, learnable, regionalized process-based models with physical outputs can approach state-of-the-art hydrologic prediction accuracy
• Prolongation of SMAP to Spatio-temporally Seamless Coverage of Continental US Using a Deep Learning Neural Network
• Physics guided machine learning using simplified theories
• A Differentiable Programming System to Bridge Machine Learning and Scientific Computing
• The Need for Ethical, Responsible, and Trustworthy Artificial Intelligence for Environmental Sciences
• PDEBENCH: An Extensive Benchmark for Scientific Machine Learning
• DeepClouds.ai: Deep learning enabled computationally cheap direct numerical simulations
• Bathymetry Inversion using a Deep-Learning-Based Surrogate for Shallow Water Equations Solvers

Cited by in corpus (6)

• When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System
• Interpretable structural model error discovery from sparse assimilation increments using spectral bias-reduced neural networks: A quasi-geostrophic turbulence test case
• Scientific machine learning in Hydrology: a unified perspective
• Scientific Machine Learning of Flow Resistance Using Universal Shallow Water Equations with Differentiable Programming
• A Comparison of Mesh-Free Differentiable Programming and Data-Driven Strategies for Optimal Control under PDE Constraints
• Neural-Parameterized Cellular Automata for Wildfire Spread