Interpretable structural model error discovery from sparse assimilation increments using spectral bias-reduced neural networks: A quasi-geostrophic turbulence test case

Abstract

Earth system models suffer from various structural and parametric errors in their representation of nonlinear, multi-scale processes, leading to uncertainties in their long-term projections. The effects of many of these errors (particularly those due to fast physics) can be quantified in short-term simulations, e.g., as differences between the predicted and observed states (analysis increments). With the increase in the availability of high-quality observations and simulations, learning nudging from these increments to correct model errors has become an active research area. However, most studies focus on using neural networks, which while powerful, are hard to interpret, are data-hungry, and poorly generalize out-of-distribution. Here, we show the capabilities of Model Error Discovery with Interpretability and Data Assimilation (MEDIDA), a general, data-efficient framework that uses sparsity-promoting equation-discovery techniques to learn model errors from analysis increments. Using two-layer quasi-geostrophic turbulence as the test case, MEDIDA is shown to successfully discover various linear and nonlinear structural/parametric errors when full observations are available. Discovery from spatially sparse observations is found to require highly accurate interpolation schemes. While NNs have shown success as interpolators in recent studies, here, they are found inadequate due to their inability to accurately represent small scales, a phenomenon known as spectral bias. We show that a general remedy, adding a random Fourier feature layer to the NN, resolves this issue enabling MEDIDA to successfully discover model errors from sparse observations. These promising results suggest that with further development, MEDIDA could be scaled up to models of the Earth system and real observations.

arXiv comment26 pages, 5+1 figures

References in corpus (24)

• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• Differentiable modeling to unify machine learning and physical models and advance Geosciences
• Stable machine-learning parameterization of subgrid processes for climate modeling at a range of resolutions
• Using machine learning to correct model error in data assimilation and forecast applications
• Using Noisy or Incomplete Data to Discover Models of Spatiotemporal Dynamics
• Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning
• Use of neural networks for stable, accurate and physically consistent parameterization of subgrid atmospheric processes with good performance at reduced precision
• Calibration and Uncertainty Quantification of Convective Parameters in an Idealized GCM
• Applying machine learning to improve simulations of a chaotic dynamical system using empirical error correction
• Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows
• A posteriori learning for quasi-geostrophic turbulence parametrization
• Reconstructing velocity and pressure from sparse noisy particle tracks using Physics-Informed Neural Networks
• Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow
• The vortex gas scaling regime of baroclinic turbulence
• Deep learning of systematic sea ice model errors from data assimilation increments
• Deep learning-enhanced ensemble-based data assimilation for high-dimensional nonlinear dynamical systems
• Discovering Sparse Interpretable Dynamics from Partial Observations
• Ensemble-based kernel learning for a class of data assimilation problems with imperfect forward simulators
• Concurrent multi-parameter learning demonstrated on the Kuramoto-Sivashinsky equation
• Ensemble flow reconstruction in the atmospheric boundary layer from spatially limited measurements through latent diffusion models
• Explainable Offline-Online Training of Neural Networks for Parameterizations: A 1D Gravity Wave-QBO Testbed in the Small-data Regime
• Discovery of interpretable structural model errors by combining Bayesian sparse regression and data assimilation: A chaotic Kuramoto-Sivashinsky test case
• Flow reconstruction by multiresolution optimization of a discrete loss with automatic differentiation
• A Causality-Based Learning Approach for Discovering the Underlying Dynamics of Complex Systems from Partial Observations with Stochastic Parameterization

Cited by in corpus (1)

• Learning Closed-form Equations for Subgrid-scale Closures from High-fidelity Data: Promises and Challenges