Neural Closure Models for Dynamical Systems

Abstract

Complex dynamical systems are used for predictions in many domains. Because of computational costs, models are truncated, coarsened, or aggregated. As the neglected and unresolved terms become important, the utility of model predictions diminishes. We develop a novel, versatile, and rigorous methodology to learn non-Markovian closure parameterizations for known-physics/low-fidelity models using data from high-fidelity simulations. The new "neural closure models" augment low-fidelity models with neural delay differential equations (nDDEs), motivated by the Mori-Zwanzig formulation and the inherent delays in complex dynamical systems. We demonstrate that neural closures efficiently account for truncated modes in reduced-order-models, capture the effects of subgrid-scale processes in coarse models, and augment the simplification of complex biological and physical-biogeochemical models. We find that using non-Markovian over Markovian closures improves long-term prediction accuracy and requires smaller networks. We derive adjoint equations and network architectures needed to efficiently implement the new discrete and distributed nDDEs, for any time-integration schemes and allowing nonuniformly-spaced temporal training data. The performance of discrete over distributed delays in closure models is explained using information theory, and we find an optimal amount of past information for a specified architecture. Finally, we analyze computational complexity and explain the limited additional cost due to neural closure models.

arXiv comment29 pages, 9 figures, 13 pages of supplementary information

References in corpus (17)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Proper Orthogonal Decomposition Closure Models For Turbulent Flows: A Numerical Comparison
• Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations
• Data-assisted reduced-order modeling of extreme events in complex dynamical systems
• Data-driven non-Markovian closure models
• Time-series learning of latent-space dynamics for reduced-order model closure
• Data-driven recovery of hidden physics in reduced order modeling of fluid flows
• Use of neural networks for stable, accurate and physically consistent parameterization of subgrid atmospheric processes with good performance at reduced precision
• Data-driven Discovery of Closure Models
• Solvable delay model for epidemic spreading: the case of Covid-19 in Italy
• A Priori Estimation Of Memory Effects In Coarse-Grained Nonlinear Systems Using The Mori-Zwanzig Formalism
• A Geometric Approach to Dynamical Model-Order Reduction
• Influence of the spatial resolution on fine-scale features in DNS of turbulence generated by a single square grid
• Variational approach to closure of nonlinear dynamical systems: Autonomous case
• Evaluation of explicit and implicit LES closures for Burgers turbulence
• Interpolation Technique to Speed Up Gradients Propagation in Neural ODEs
• Mori-Zwanzig reduced models for uncertainty quantification

Cited by in corpus (10)

• A Review of Machine Learning Methods Applied to Structural Dynamics and Vibroacoustic
• On closures for reduced order models $-$ A spectrum of first-principle to machine-learned avenues
• Super-resolution data assimilation
• Theoretical tools for understanding the climate crisis from Hasselmann's program and beyond
• Comparison of neural closure models for discretised PDEs
• Neural Dynamical Operator: Continuous Spatial-Temporal Model with Gradient-Based and Derivative-Free Optimization Methods
• Bounded nonlinear forecasts of partially observed geophysical systems with physics-constrained deep learning
• Evaluation of Deep Neural Operator Models toward Ocean Forecasting
• Guided Unconditional and Conditional Generative Models for Super-Resolution and Inference of Quasi-Geostrophic Turbulence
• Evaluation of Analytical Turbulence Closures for Quasi-Geostrophic Ocean Flows with Coastal Boundaries