Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems

Abstract

The process of transforming observed data into predictive mathematical models of the physical world has always been paramount in science and engineering. Although data is currently being collected at an ever-increasing pace, devising meaningful models out of such observations in an automated fashion still remains an open problem. In this work, we put forth a machine learning approach for identifying nonlinear dynamical systems from data. Specifically, we blend classical tools from numerical analysis, namely the multi-step time-stepping schemes, with powerful nonlinear function approximators, namely deep neural networks, to distill the mechanisms that govern the evolution of a given data-set. We test the effectiveness of our approach for several benchmark problems involving the identification of complex, nonlinear and chaotic dynamics, and we demonstrate how this allows us to accurately learn the dynamics, forecast future states, and identify basins of attraction. In particular, we study the Lorenz system, the fluid flow behind a cylinder, the Hopf bifurcation, and the Glycoltic oscillator model as an example of complicated nonlinear dynamics typical of biological systems.

References in corpus (5)

• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations
• Multi-level Residual Networks from Dynamical Systems View
• Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations

Cited by in corpus (73)

• Data-driven discovery of coordinates and governing equations
• Neural Ordinary Differential Equations
• Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• Data Driven Governing Equations Approximation Using Deep Neural Networks
• Neural Granger Causality
• Understanding and mitigating gradient pathologies in physics-informed neural networks
• Deep learning of dynamics and signal-noise decomposition with time-stepping constraints
• A deep learning enabler for non-intrusive reduced order modeling of fluid flows
• Discovery of Nonlinear Multiscale Systems: Sampling Strategies and Embeddings
• Data-Driven Deep Learning of Partial Differential Equations in Modal Space
• Deep learning algorithm for data-driven simulation of noisy dynamical system
• Neural Jump Stochastic Differential Equations
• A mesh-free method for interface problems using the deep learning approach
• SPINN: Sparse, Physics-based, and partially Interpretable Neural Networks for PDEs
• A non-intrusive reduced order modeling framework for quasi-geostrophic turbulence
• Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data
• Active Training of Physics-Informed Neural Networks to Aggregate and Interpolate Parametric Solutions to the Navier-Stokes Equations
• Discovery of Nonlinear Dynamical Systems using a Runge-Kutta Inspired Dictionary-based Sparse Regression Approach
• GFINNs: GENERIC Formalism Informed Neural Networks for Deterministic and Stochastic Dynamical Systems
• Data-Driven Reduced-Order Modeling of Spatiotemporal Chaos with Neural Ordinary Differential Equations
• On spike-and-slab priors for Bayesian equation discovery of nonlinear dynamical systems via sparse linear regression
• Memory embedded non-intrusive reduced order modeling of non-ergodic flows
• On generalized residue network for deep learning of unknown dynamical systems
• Neural Network Augmented Physics Models for Systems with Partially Unknown Dynamics: Application to Slider-Crank Mechanism
• OnsagerNet: Learning Stable and Interpretable Dynamics using a Generalized Onsager Principle
• Learning interaction kernels in heterogeneous systems of agents from multiple trajectories
• Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profiles
• Learning Latent Dynamics for Partially-Observed Chaotic Systems
• Learning elliptic partial differential equations with randomized linear algebra
• Using neural networks to accelerate the solution of the Boltzmann equation
• A comparative study of physics-informed neural network models for learning unknown dynamics and constitutive relations
• Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks
• Data-driven discovery of free-form governing differential equations
• Knowledge-Based Learning of Nonlinear Dynamics and Chaos
• EM-like Learning Chaotic Dynamics from Noisy and Partial Observations
• Data-driven Identification of 2D Partial Differential Equations using extracted physical features
• Data-driven Prediction of General Hamiltonian Dynamics via Learning Exactly-Symplectic Maps
• Deep Hamiltonian networks based on symplectic integrators
• Hypersolvers: Toward Fast Continuous-Depth Models
• A General Framework for Structured Learning of Mechanical Systems
• Deep Euler method: solving ODEs by approximating the local truncation error of the Euler method
• 3D Shape Synthesis for Conceptual Design and Optimization Using Variational Autoencoders
• HyperPINN: Learning parameterized differential equations with physics-informed hypernetworks
• Machine Discovery of Partial Differential Equations from Spatiotemporal Data
• Discovery of Dynamics Using Linear Multistep Methods
• Bridging the Gap: Machine Learning to Resolve Improperly Modeled Dynamics
• Recurrent Neural Networks for Partially Observed Dynamical Systems
• A Data Driven Method for Computing Quasipotentials
• Global and Local Reduced Models for Interacting, Heterogeneous Agents
• Inverse modified differential equations for discovery of dynamics
• Variational Deep Learning for the Identification and Reconstruction of Chaotic and Stochastic Dynamical Systems from Noisy and Partial Observations
• Learning Dynamics from Noisy Measurements using Deep Learning with a Runge-Kutta Constraint
• Data-driven learning of non-autonomous systems
• Discovering time-varying aeroelastic models of a long-span suspension bridge from field measurements by sparse identification of nonlinear dynamical systems
• The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation
• Deep Neural Network Modeling of Unknown Partial Differential Equations in Nodal Space
• SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems
• Discovery of Governing Equations with Recursive Deep Neural Networks
• STENCIL-NET: Data-driven solution-adaptive discretization of partial differential equations
• Meta-Solver for Neural Ordinary Differential Equations
• Learning Runge-Kutta Integration Schemes for ODE Simulation and Identification
• Nested Mixture of Experts: Cooperative and Competitive Learning of Hybrid Dynamical System
• A Neural Network Ensemble Approach to System Identification
• Deep learning of parameterized equations with applications to uncertainty quantification
• Accuracy and Architecture Studies of Residual Neural Network solving Ordinary Differential Equations
• Learning Interaction Kernels for Agent Systems on Riemannian Manifolds
• Non-intrusive inference reduced order model for fluids using linear multistep neural network
• Invertible generalized synchronization: A putative mechanism for implicit learning in biological and artificial neural systems
• Multiscale and Nonlocal Learning for PDEs using Densely Connected RNNs
• Hamiltonian-based Neural ODE Networks on the SE(3) Manifold For Dynamics Learning and Control
• Reduced Order Modeling using Shallow ReLU Networks with Grassmann Layers
• MAGI-X: Manifold-Constrained Gaussian Process Inference for Unknown System Dynamics