CGKN: A Deep Learning Framework for Modeling Complex Dynamical Systems and Efficient Data Assimilation

Abstract

Deep learning is widely used to predict complex dynamical systems in many scientific and engineering areas. However, the black-box nature of these deep learning models presents significant challenges for carrying out simultaneous data assimilation (DA), which is a crucial technique for state estimation, model identification, and reconstructing missing data. Integrating ensemble-based DA methods with nonlinear deep learning models is computationally expensive and may suffer from large sampling errors. To address these challenges, we introduce a deep learning framework designed to simultaneously provide accurate forecasts and efficient DA. It is named Conditional Gaussian Koopman Network (CGKN), which transforms general nonlinear systems into nonlinear neural differential equations with conditional Gaussian structures. CGKN aims to retain essential nonlinear components while applying systematic and minimal simplifications to facilitate the development of analytic formulae for nonlinear DA. This allows for seamless integration of DA performance into the deep learning training process, eliminating the need for empirical tuning as required in ensemble methods. CGKN compensates for structural simplifications by lifting the dimension of the system, which is motivated by Koopman theory. Nevertheless, CGKN exploits special nonlinear dynamics within the lifted space. This enables the model to capture extreme events and strong non-Gaussian features in joint and marginal distributions with appropriate uncertainty quantification. We demonstrate the effectiveness of CGKN for both prediction and DA on three strongly nonlinear and non-Gaussian turbulent systems: the projected stochastic Burgers-Sivashinsky equation, the Lorenz 96 system, and the El Niño-Southern Oscillation. The results justify the robustness and computational efficiency of CGKN.

References in corpus (32)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data
• Turbulence Modeling in the Age of Data
• Surrogate Modeling for Fluid Flows Based on Physics-Constrained Deep Learning Without Simulation Data
• Applied Koopmanism
• The GNAT method for nonlinear model reduction: effective implementation and application to computational fluid dynamics and turbulent flows
• A Physics Informed Machine Learning Approach for Reconstructing Reynolds Stress Modeling Discrepancies Based on DNS Data
• KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics
• A Comprehensive Physics-Informed Machine Learning Framework for Predictive Turbulence Modeling
• Physics-Informed Machine Learning Approach for Augmenting Turbulence Models: A Comprehensive Framework
• Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders
• Machine Learning for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz '96 Model
• Sparse Dynamics for Partial Differential Equations
• Extensive Chaos in the Lorenz-96 Model
• Physics-Informed Probabilistic Learning of Linear Embeddings of Non-linear Dynamics With Guaranteed Stability
• Time-series learning of latent-space dynamics for reduced-order model closure
• How Entropic Regression Beats the Outliers Problem in Nonlinear System Identification
• Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations
• Learning nonlocal constitutive models with neural networks
• Using Data Assimilation to Train a Hybrid Forecast System that Combines Machine-Learning and Knowledge-Based Components
• State, global and local parameter estimation using local ensemble Kalman filters: applications to online machine learning of chaotic dynamics
• Machine learning with observers predicts complex spatiotemporal behavior
• Conditional Gaussian Nonlinear System: a Fast Preconditioner and a Cheap Surrogate Model For Complex Nonlinear Systems
• Nonlinear Discrete-Time Observers with Physics-Informed Neural Networks
• BAMCAFE: A Bayesian Machine Learning Advanced Forecast Ensemble Method for Complex Turbulent Systems with Partial Observations
• A Causality-Based Learning Approach for Discovering the Underlying Dynamics of Complex Systems from Partial Observations with Stochastic Parameterization
• Neural Dynamical Operator: Continuous Spatial-Temporal Model with Gradient-Based and Derivative-Free Optimization Methods
• Entropic Causal Inference for Neurological Applications
• CEBoosting: Online Sparse Identification of Dynamical Systems with Regime Switching by Causation Entropy Boosting
• CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilation
• An equivariant neural operator for developing nonlocal tensorial constitutive models

Cited by in corpus (3)

• Modeling Partially Observed Nonlinear Dynamical Systems and Efficient Data Assimilation via Discrete-Time Conditional Gaussian Koopman Network
• Uncertainty Propagation Networks for Neural Ordinary Differential Equations
• Active Learning of Model Discrepancy with Bayesian Experimental Design