Long-time predictive modeling of nonlinear dynamical systems using neural networks

Abstract

We study the use of feedforward neural networks (FNN) to develop models of nonlinear dynamical systems from data. Emphasis is placed on predictions at long times, with limited data availability. Inspired by global stability analysis, and the observation of the strong correlation between the local error and the maximum singular value of the Jacobian of the ANN, we introduce Jacobian regularization in the loss function. This regularization suppresses the sensitivity of the prediction to the local error and is shown to improve accuracy and robustness. Comparison between the proposed approach and sparse polynomial regression is presented in numerical examples ranging from simple ODE systems to nonlinear PDE systems including vortex shedding behind a cylinder, and instability-driven buoyant mixing flow. Furthermore, limitations of feedforward neural networks are highlighted, especially when the training data does not include a low dimensional attractor. Strategies of data augmentation are presented as remedies to address these issues to a certain extent.

arXiv comment30 pages. Complexity, 2018

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Practical Bayesian Optimization of Machine Learning Algorithms
• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Adding noise to the input of a model trained with a regularized objective
• Revise Saturated Activation Functions

Cited by in corpus (26)

• Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks
• Deep learning of dynamics and signal-noise decomposition with time-stepping constraints
• Construction of Reduced Order Models for Fluid Flows Using Deep Feedforward Neural Networks
• Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning
• Assessment of unsteady flow predictions using hybrid deep learning based reduced order models
• Deconvolutional artificial neural network models for large eddy simulation of turbulence
• On the Structure of Time-delay Embedding in Linear Models of Non-linear Dynamical Systems
• Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction
• Data-driven subgrid-scale modeling of forced Burgers turbulence using deep learning with generalization to higher Reynolds numbers via transfer learning
• Sparsity-promoting algorithms for the discovery of informative Koopman invariant subspaces
• Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES
• Equation-free surrogate modeling of geophysical flows at the intersection of machine learning and data assimilation
• Model Order Reduction based on Runge-Kutta Neural Network
• Comparison of neural closure models for discretised PDEs
• Calibration of projection-based reduced-order models for unsteady compressible flows
• Initializing LSTM internal states via manifold learning
• Learning Dynamics from Noisy Measurements using Deep Learning with a Runge-Kutta Constraint
• Polynomial Neural Networks and Taylor maps for Dynamical Systems Simulation and Learning
• Differentiable programming and its applications to dynamical systems
• Tensorized Transformer for Dynamical Systems Modeling
• Matrix Lie Maps and Neural Networks for Solving Differential Equations
• A Model-Constrained Tangent Slope Learning Approach for Dynamical Systems
• Assessment of End-to-End and Sequential Data-driven Learning of Fluid Flows
• TAEN: A Model-Constrained Tikhonov Autoencoder Network for Forward and Inverse Problems
• Learning emergent PDEs in a learned emergent space
• Latent-Lagrangian Neural Networks for Reduced Order Modeling of Non-autonomous Nonlinear Dynamical Systems