CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilation

Abstract

A new knowledge-based and machine learning hybrid modeling approach, called conditional Gaussian neural stochastic differential equation (CGNSDE), is developed to facilitate modeling complex dynamical systems and implementing analytic formulae of the associated data assimilation (DA). In contrast to the standard neural network predictive models, the CGNSDE is designed to effectively tackle both forward prediction tasks and inverse state estimation problems. The CGNSDE starts by exploiting a systematic causal inference via information theory to build a simple knowledge-based nonlinear model that nevertheless captures as much explainable physics as possible. Then, neural networks are supplemented to the knowledge-based model in a specific way, which not only characterizes the remaining features that are challenging to model with simple forms but also advances the use of analytic formulae to efficiently compute the nonlinear DA solution. These analytic formulae are used as an additional computationally affordable loss to train the neural networks that directly improve the DA accuracy. This DA loss function promotes the CGNSDE to capture the interactions between state variables and thus advances its modeling skills. With the DA loss, the CGNSDE is more capable of estimating extreme events and quantifying the associated uncertainty. Furthermore, crucial physical properties in many complex systems, such as the translate-invariant local dependence of state variables, can significantly simplify the neural network structures and facilitate the CGNSDE to be applied to high-dimensional systems. Numerical experiments based on chaotic systems with intermittency and strong non-Gaussian features indicate that the CGNSDE outperforms knowledge-based regression models, and the DA loss further enhances the modeling skills of the CGNSDE.

References in corpus (28)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Machine Learning for Fluid Mechanics
• Turbulence Modeling in the Age of Data
• The GNAT method for nonlinear model reduction: effective implementation and application to computational fluid dynamics and turbulent flows
• A Physics Informed Machine Learning Approach for Reconstructing Reynolds Stress Modeling Discrepancies Based on DNS Data
• KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics
• A Comprehensive Physics-Informed Machine Learning Framework for Predictive Turbulence Modeling
• Physics-Informed Machine Learning Approach for Augmenting Turbulence Models: A Comprehensive Framework
• Stochastic Parameterization: Towards a new view of Weather and Climate Models
• Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems
• Data-Driven POD-Galerkin Reduced Order Model for Turbulent Flows
• Machine Learning for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz '96 Model
• Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model
• On closures for reduced order models $-$ A spectrum of first-principle to machine-learned avenues
• Data-Driven Variational Multiscale Reduced Order Models
• Data-driven model reduction, Wiener projections, and the Koopman-Mori-Zwanzig formalism
• How Entropic Regression Beats the Outliers Problem in Nonlinear System Identification
• Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations
• Efficient Statistically Accurate Algorithms for the Fokker-Planck Equation in Large Dimensions
• Reduced-Order Models for Coupled Dynamical Systems: Data-driven Methods and the Koopman Operator
• Data-adaptive harmonic spectra and multilayer Stuart-Landau models
• Deep learning-enhanced ensemble-based data assimilation for high-dimensional nonlinear dynamical systems
• Using Data Assimilation to Train a Hybrid Forecast System that Combines Machine-Learning and Knowledge-Based Components
• State, global and local parameter estimation using local ensemble Kalman filters: applications to online machine learning of chaotic dynamics
• Conditional Gaussian Nonlinear System: a Fast Preconditioner and a Cheap Surrogate Model For Complex Nonlinear Systems
• An Efficient and Statistically Accurate Lagrangian Data Assimilation Algorithm with Applications to Discrete Element Sea Ice Models
• A Causality-Based Learning Approach for Discovering the Underlying Dynamics of Complex Systems from Partial Observations with Stochastic Parameterization
• CEBoosting: Online Sparse Identification of Dynamical Systems with Regime Switching by Causation Entropy Boosting

Cited by in corpus (7)

• Neural Dynamical Operator: Continuous Spatial-Temporal Model with Gradient-Based and Derivative-Free Optimization Methods
• CGKN: A Deep Learning Framework for Modeling Complex Dynamical Systems and Efficient Data Assimilation
• Modeling Partially Observed Nonlinear Dynamical Systems and Efficient Data Assimilation via Discrete-Time Conditional Gaussian Koopman Network
• Active Learning of Model Discrepancy with Bayesian Experimental Design
• Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance
• Bayesian Experimental Design for Model Discrepancy Calibration: An Auto-Differentiable Ensemble Kalman Inversion Approach
• A Martingale-Free Introduction to Conditional Gaussian Nonlinear Systems