Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems

Abstract

In data-driven modeling of spatiotemporal phenomena careful consideration often needs to be made in capturing the dynamics of the high wavenumbers. This problem becomes especially challenging when the system of interest exhibits shocks or chaotic dynamics. We present a data-driven modeling method that accurately captures shocks and chaotic dynamics by proposing a novel architecture, stabilized neural ordinary differential equation (ODE). In our proposed architecture, we learn the right-hand-side (RHS) of an ODE by adding the outputs of two NN together where one learns a linear term and the other a nonlinear term. Specifically, we implement this by training a sparse linear convolutional NN to learn the linear term and a dense fully-connected nonlinear NN to learn the nonlinear term. This is in contrast with the standard neural ODE which involves training only a single NN for learning the RHS. We apply this setup to the viscous Burgers equation, which exhibits shocked behavior, and show better short-time tracking and prediction of the energy spectrum at high wavenumbers than a standard neural ODE. We also find that the stabilized neural ODE models are much more robust to noisy initial conditions than the standard neural ODE approach. We also apply this method to chaotic trajectories of the Kuramoto-Sivashinsky equation. In this case, stabilized neural ODEs keep long-time trajectories on the attractor, and are highly robust to noisy initial conditions, while standard neural ODEs fail at achieving either of these results. We conclude by demonstrating how stabilizing neural ODEs provide a natural extension for use in reduced-order modeling by projecting the dynamics onto the eigenvectors of the learned linear term.

References in corpus (2)

• Applied Koopmanism
• Stationary two-dimensional turbulence statistics using a Markovian forcing scheme

Cited by in corpus (15)

• Deep Learning for Time Series Forecasting: A Survey
• Compressing fluid flows with nonlinear machine learning: mode decomposition, latent modeling, and flow control
• Deep learning delay coordinate dynamics for chaotic attractors from partial observable data
• A Multifidelity deep operator network approach to closure for multiscale systems
• Data-driven state-space and Koopman operator models of coherent state dynamics on invariant manifolds
• Model-Based Reinforcement Learning for Control of Strongly-Disturbed Unsteady Aerodynamic Flows
• Neural Dynamical Operator: Continuous Spatial-Temporal Model with Gradient-Based and Derivative-Free Optimization Methods
• Physics-agnostic and Physics-infused machine learning for thin films flows: modeling, and predictions from small data
• Real-time forecasting of chaotic dynamics from sparse data and autoencoders
• Dynamics of a Data-Driven Low-Dimensional Model of Turbulent Minimal Pipe Flow
• Elastoinertial turbulence: Data-driven reduced-order model based on manifold dynamics
• Enhancing Dynamical System Modeling through Interpretable Machine Learning Augmentations: A Case Study in Cathodic Electrophoretic Deposition
• Non-intrusive model combination for learning dynamical systems
• Data-driven discovery of dynamics from time-resolved coherent scattering
• Invariant Measures in Time-Delay Coordinates for Unique Dynamical System Identification