Learning Closed-form Equations for Subgrid-scale Closures from High-fidelity Data: Promises and Challenges

Abstract

There is growing interest in discovering interpretable, closed-form equations for subgrid-scale (SGS) closures/parameterizations of complex processes in Earth systems. Here, we apply a common equation-discovery technique with expansive libraries to learn closures from filtered direct numerical simulations of 2D turbulence and Rayleigh-Bénard convection (RBC). Across common filters (e.g., Gaussian, box), we robustly discover closures of the same form for momentum and heat fluxes. These closures depend on nonlinear combinations of gradients of filtered variables, with constants that are independent of the fluid/flow properties and only depend on filter type/size. We show that these closures are the nonlinear gradient model (NGM), which is derivable analytically using Taylor-series. Indeed, we suggest that with common (physics-free) equation-discovery algorithms, for many common systems/physics, discovered closures are consistent with the leading term of the Taylor-series (except when cutoff filters are used). Like previous studies, we find that large-eddy simulations with NGM closures are unstable, despite significant similarities between the true and NGM-predicted fluxes (correlations $>0.95$). We identify two shortcomings as reasons for these instabilities: in 2D, NGM produces zero kinetic energy transfer between resolved and subgrid scales, lacking both diffusion and backscattering. In RBC, potential energy backscattering is poorly predicted. Moreover, we show that SGS fluxes diagnosed from data, presumed the ''truth'' for discovery, depend on filtering procedures and are not unique. Accordingly, to learn accurate, stable closures in future work, we propose several ideas around using physics-informed libraries, loss functions, and metrics. These findings are relevant to closure modeling of any multi-scale system.

40 pages, 4 figures. The code for 2D-FHIT solver "py2d" is available at https://github.com/envfluids/py2d. The code and data used for analysis in this work can be found at https://github.com/jakharkaran/EqsDiscovery_2D-FHIT_RBC and https://doi.org/10.5281/zenodo.7500647, respectively

References in corpus (39)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Machine Learning for Fluid Mechanics
• Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
• Enforcing Analytic Constraints in Neural-Networks Emulating Physical Systems
• Stochastic Parameterization: Towards a new view of Weather and Climate Models
• Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations
• Stable machine-learning parameterization of subgrid processes for climate modeling at a range of resolutions
• Sub-grid modelling for two-dimensional turbulence using neural networks
• Perspectives on Machine Learning-augmented Reynolds-averaged and Large Eddy Simulation Models of Turbulence
• Discovering Symbolic Models from Deep Learning with Inductive Biases
• Model selection for dynamical systems via sparse regression and information criteria
• Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl
• Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning
• Multi-level Dynamical Systems: Connecting the Ruelle Response Theory and the Mori-Zwanzig Approach
• Any equation is a forest: Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE)
• Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience
• Wall to Wall Optimal Transport
• Discovery of Nonlinear Dynamical Systems using a Runge-Kutta Inspired Dictionary-based Sparse Regression Approach
• Data-driven subgrid-scale modeling of forced Burgers turbulence using deep learning with generalization to higher Reynolds numbers via transfer learning
• A posteriori learning for quasi-geostrophic turbulence parametrization
• Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES
• Non-Markovian Closure Models for Large Eddy Simulations using the Mori-Zwanzig Formalism
• Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow
• Explainable Artificial Intelligence for Bayesian Neural Networks: Towards trustworthy predictions of ocean dynamics
• The vortex gas scaling regime of baroclinic turbulence
• Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96 systems and transfer-learning
• Optimal Heat Transport in Rayleigh-Bénard Convection
• Climbing down Charney's ladder: Machine Learning and the post-Dennard era of computational climate science
• Data-driven reduced modelling of turbulent Rayleigh-Benard convection using DMD-enhanced Fluctuation-Dissipation Theorem
• Explainable Offline-Online Training of Neural Networks for Parameterizations: A 1D Gravity Wave-QBO Testbed in the Small-data Regime
• Large-eddy simulations with ClimateMachine: a new open-source code for atmospheric simulations on GPUs and CPUs
• Discovery of interpretable structural model errors by combining Bayesian sparse regression and data assimilation: A chaotic Kuramoto-Sivashinsky test case
• Effects of resolution inhomogeneity in large-eddy simulation
• Coherent Solutions and Transition to Turbulence in Two-Dimensional Rayleigh-Bénard Convection
• Subgrid parameterizations of ocean mesoscale eddies based on Germano decomposition
• Reduced-order modeling of fully turbulent buoyancy-driven flows using the Green's function method
• Optimal Clipping of Structural Subgrid Stress Closures for Large Eddy Simulation
• Interpretable structural model error discovery from sparse assimilation increments using spectral bias-reduced neural networks: A quasi-geostrophic turbulence test case
• Turbulence closure with small, local neural networks: Forced two-dimensional and $\beta$-plane flows

Cited by in corpus (3)

• A stable implementation of a data-driven scale-aware mesoscale parameterization
• Guided Unconditional and Conditional Generative Models for Super-Resolution and Inference of Quasi-Geostrophic Turbulence
• Evaluation of Analytical Turbulence Closures for Quasi-Geostrophic Ocean Flows with Coastal Boundaries