Neural Dynamical Operator: Continuous Spatial-Temporal Model with Gradient-Based and Derivative-Free Optimization Methods

Abstract

Data-driven modeling techniques have been explored in the spatial-temporal modeling of complex dynamical systems for many engineering applications. However, a systematic approach is still lacking to leverage the information from different types of data, e.g., with different spatial and temporal resolutions, and the combined use of short-term trajectories and long-term statistics. In this work, we build on the recent progress of neural operator and present a data-driven modeling framework called neural dynamical operator that is continuous in both space and time. A key feature of the neural dynamical operator is the resolution-invariance with respect to both spatial and temporal discretizations, without demanding abundant training data in different temporal resolutions. To improve the long-term performance of the calibrated model, we further propose a hybrid optimization scheme that leverages both gradient-based and derivative-free optimization methods and efficiently trains on both short-term time series and long-term statistics. We investigate the performance of the neural dynamical operator with three numerical examples, including the viscous Burgers' equation, the Navier-Stokes equations, and the Kuramoto-Sivashinsky equation. The results confirm the resolution-invariance of the proposed modeling framework and also demonstrate stable long-term simulations with only short-term time series data. In addition, we show that the proposed model can better predict long-term statistics via the hybrid optimization scheme with a combined use of short-term and long-term data.

References in corpus (28)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Traffic and Related Self-Driven Many-Particle Systems
• Machine Learning for Fluid Mechanics
• Turbulence Modeling in the Age of Data
• A Physics Informed Machine Learning Approach for Reconstructing Reynolds Stress Modeling Discrepancies Based on DNS Data
• A Comprehensive Physics-Informed Machine Learning Framework for Predictive Turbulence Modeling
• Physics-Informed Machine Learning Approach for Augmenting Turbulence Models: A Comprehensive Framework
• The Ensemble Kalman Filter for Inverse Problems
• Burgers Turbulence
• A Review of automatic differentiation and its efficient implementation
• Multipole Graph Neural Operator for Parametric Partial Differential Equations
• Least Squares Shadowing sensitivity analysis of chaotic limit cycle oscillations
• Neural Operator: Graph Kernel Network for Partial Differential Equations
• Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks
• Time-series learning of latent-space dynamics for reduced-order model closure
• Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems
• Regularized Ensemble Kalman Methods for Inverse Problems
• Multiwavelet-based Operator Learning for Differential Equations
• Ensemble Kalman Methods With Constraints
• Data-Driven Reduced-Order Modeling of Spatiotemporal Chaos with Neural Ordinary Differential Equations
• In-Context Operator Learning with Data Prompts for Differential Equation Problems
• Ensemble Kalman Inversion: mean-field limit and convergence analysis
• Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems
• Neural Closure Models for Dynamical Systems
• A Causality-Based Learning Approach for Discovering the Underlying Dynamics of Complex Systems from Partial Observations with Stochastic Parameterization
• CEBoosting: Online Sparse Identification of Dynamical Systems with Regime Switching by Causation Entropy Boosting
• CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilation

Cited by in corpus (6)

• CGKN: A Deep Learning Framework for Modeling Complex Dynamical Systems and Efficient Data Assimilation
• Modeling Partially Observed Nonlinear Dynamical Systems and Efficient Data Assimilation via Discrete-Time Conditional Gaussian Koopman Network
• Active Learning of Model Discrepancy with Bayesian Experimental Design
• Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance
• Bayesian Experimental Design for Model Discrepancy Calibration: An Auto-Differentiable Ensemble Kalman Inversion Approach
• Operator learning for models of tear film breakup