On generalized residue network for deep learning of unknown dynamical systems

Abstract

We present a general numerical approach for learning unknown dynamical systems using deep neural networks (DNNs). Our method is built upon recent studies that identified the residue network (ResNet) as an effective neural network structure. In this paper, we present a generalized ResNet framework and broadly define residue as the discrepancy between observation data and prediction made by another model, which can be an existing coarse model or reduced-order model. In this case, the generalized ResNet serves as a model correction to the existing model and recovers the unresolved dynamics. When an existing coarse model is not available, we present numerical strategies for fast creation of coarse models, to be used in conjunction with the generalized ResNet. These coarse models are constructed using the same data set and thus do not require additional resources. The generalized ResNet is capable of learning the underlying unknown equations and producing predictions with accuracy higher than the standard ResNet structure. This is demonstrated via several numerical examples, including long-term prediction of a chaotic system.

References in corpus (22)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Solving high-dimensional partial differential equations using deep learning
• On Dynamic Mode Decomposition: Theory and Applications
• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• Chaos as an Intermittently Forced Linear System
• Machine Learning of Linear Differential Equations using Gaussian Processes
• PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network
• Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations
• Data Driven Governing Equations Approximation Using Deep Neural Networks
• PDE-Net: Learning PDEs from Data
• Deep learning of dynamics and signal-noise decomposition with time-stepping constraints
• A deep learning enabler for non-intrusive reduced order modeling of fluid flows
• Simulator-free Solution of High-Dimensional Stochastic Elliptic Partial Differential Equations using Deep Neural Networks
• Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems
• A machine learning approach for efficient uncertainty quantification using multiscale methods
• Numerical Aspects for Approximating Governing Equations Using Data
• EM-like Learning Chaotic Dynamics from Noisy and Partial Observations
• IDENT: Identifying Differential Equations with Numerical Time evolution

Cited by in corpus (9)

• Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows
• Accelerating hydrodynamic simulations of urban drainage systems with physics-guided machine learning
• Approaching epidemiological dynamics of COVID-19 with physics-informed neural networks
• Bayesian System ID: Optimal management of parameter, model, and measurement uncertainty
• Extended Dynamic Mode Decomposition for Inhomogeneous Problems
• Data-driven learning of non-autonomous systems
• Discovery of Governing Equations with Recursive Deep Neural Networks
• Critical Sampling for Robust Evolution Operator Learning of Unknown Dynamical Systems
• Hard-constraint physics-residual networks for hydrogen crossover prediction and high-pressure extrapolation in PEM water electrolysis