Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks

Abstract

We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dynamical systems in their reduced order space and are shown to be an effective set of nonlinear approximators of their attractor. We demonstrate the forecasting performance of the LSTM and compare it with Gaussian processes (GPs) in time series obtained from the Lorenz 96 system, the Kuramoto-Sivashinsky equation and a prototype climate model. The LSTM networks outperform the GPs in short-term forecasting accuracy in all applications considered. A hybrid architecture, extending the LSTM with a mean stochastic model (MSM-LSTM), is proposed to ensure convergence to the invariant measure. This novel hybrid method is fully data-driven and extends the forecasting capabilities of LSTM networks.

31 pages

References in corpus (13)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• A Data-Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
• On Dynamic Mode Decomposition: Theory and Applications
• Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data
• Ergodic theory, Dynamic Mode Decomposition and Computation of Spectral Properties of the Koopman operator
• Prediction of Dynamical Systems by Symbolic Regression
• Quantification and prediction of extreme events in a one-dimensional nonlinear dispersive wave model
• Dynamical indicators for the prediction of bursting phenomena in high-dimensional systems
• Reduced models in chemical kinetics via nonlinear data-mining
• Reduced-space Gaussian Process Regression for Data-Driven Probabilistic Forecast of Chaotic Dynamical Systems
• Least Squares Shadowing Sensitivity Analysis of a Modified Kuramoto-Sivashinsky Equation
• Data-driven prediction strategies for low-frequency patterns of North Pacific climate variability
• Empirical correction of a toy climate model

Cited by in corpus (86)

• Machine Learning for Fluid Mechanics
• Data-driven discovery of coordinates and governing equations
• Enhancing Computational Fluid Dynamics with Machine Learning
• Deep Learning of Vortex Induced Vibrations
• Attractor Reconstruction by Machine Learning
• Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders
• SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamics
• Shallow Neural Networks for Fluid Flow Reconstruction with Limited Sensors
• Data-assisted reduced-order modeling of extreme events in complex dynamical systems
• Analog forecasting of extreme-causing weather patterns using deep learning
• A test case for application of convolutional neural networks to spatio-temporal climate data: Re-identifying clustered weather patterns
• PySINDy: A comprehensive Python package for robust sparse system identification
• Deep learning of dynamics and signal-noise decomposition with time-stepping constraints
• On Explaining the Surprising Success of Reservoir Computing Forecaster of Chaos? The Universal Machine Learning Dynamical System with Contrasts to VAR and DMD
• A deep learning enabler for non-intrusive reduced order modeling of fluid flows
• Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM
• Deep Learning in Deterministic Computational Mechanics
• Probabilistic neural networks for fluid flow surrogate modeling and data recovery
• On closures for reduced order models $-$ A spectrum of first-principle to machine-learned avenues
• Synthetic turbulent inflow generator using machine learning
• Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning
• Hamiltonian neural networks for solving equations of motion
• Data-driven discovery of intrinsic dynamics
• Deep learning to discover and predict dynamics on an inertial manifold
• Uniformly Accurate Machine Learning Based Hydrodynamic Models for Kinetic Equations
• A non-intrusive reduced order modeling framework for quasi-geostrophic turbulence
• LaSDI: Parametric Latent Space Dynamics Identification
• Machine-learning inference of fluid variables from data using reservoir computing
• Latent Space Data Assimilation by using Deep Learning
• Reservoir computing model of two-dimensional turbulent convection
• A long short-term memory embedding for hybrid uplifted reduced order models
• Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows
• Data-Driven Reduced-Order Modeling of Spatiotemporal Chaos with Neural Ordinary Differential Equations
• Synthetic Lagrangian Turbulence by Generative Diffusion Models
• Memory embedded non-intrusive reduced order modeling of non-ergodic flows
• Model fusion with physics-guided machine learning
• Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96 systems and transfer-learning
• Separation of Chaotic Signals by Reservoir Computing
• Reservoir Computing Approach to Quantum State Measurement
• Extreme events in globally coupled chaotic maps
• Latent-space time evolution of non-intrusive reduced-order models using Gaussian process emulation
• Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformers
• Optimized ensemble deep learning framework for scalable forecasting of dynamics containing extreme events
• Accuracy of neural networks for the simulation of chaotic dynamics: precision of training data vs precision of the algorithm
• Reservoir Computing Benchmarks: a tutorial review and critique
• Time Series Forecasting Using Manifold Learning
• Predicting Critical Transitions in Multiscale Dynamical Systems Using Reservoir Computing
• Stability analysis of chaotic systems from data
• Kernel-based Prediction of Non-Markovian Time Series
• Machine learning based non-Newtonian fluid model with molecular fidelity
• Data-driven nonintrusive reduced order modeling for dynamical systems with moving boundaries using Gaussian process regression
• Equation-free surrogate modeling of geophysical flows at the intersection of machine learning and data assimilation
• Extreme events prediction from nonlocal partial information in a spatiotemporally chaotic microcavity laser
• Discovery of interpretable structural model errors by combining Bayesian sparse regression and data assimilation: A chaotic Kuramoto-Sivashinsky test case
• Deep learning delay coordinate dynamics for chaotic attractors from partial observable data
• Data-driven Identification of 2D Partial Differential Equations using extracted physical features
• Generative Learning of the Solution of Parametric Partial Differential Equations Using Guided Diffusion Models and Virtual Observations
• Predicting Spatio-Temporal Time Series Using Dimension Reduced Local States
• Machine learning with observers predicts complex spatiotemporal behavior
• Effective models and predictability of chaotic multiscale systems via machine learning
• Reconstruction, forecasting, and stability of chaotic dynamics from partial data
• Data-driven construction of stochastic reduced dynamics encoded with non-Markovian features
• Machine learning based data-driven discovery of nonlinear phase-field dynamics
• CD-ROM: Complemented Deep-Reduced Order Model
• An Efficient Forecasting Approach to Reduce Boundary Effects in Real-Time Time-Frequency Analysis
• BAMCAFE: A Bayesian Machine Learning Advanced Forecast Ensemble Method for Complex Turbulent Systems with Partial Observations
• Machine Learning-Based Statistical Closure Models for Turbulent Dynamical Systems
• Physics-agnostic and Physics-infused machine learning for thin films flows: modeling, and predictions from small data
• Hybrid quadrature moment method for accurate and stable representation of non-Gaussian processes and their dynamics
• Validation and parameterization of a novel physics-constrained neural dynamics model applied to turbulent fluid flow
• Predicting high-dimensional heterogeneous time series employing generalized local states
• Analysis of chaotic dynamical systems with autoencoders
• Initializing LSTM internal states via manifold learning
• Global and Local Reduced Models for Interacting, Heterogeneous Agents
• Generalization capabilities and robustness of hybrid models grounded in physics compared to purely deep learning models
• SPADE4: Sparsity and Delay Embedding based Forecasting of Epidemics
• Reduction Methods in Climate Dynamics -- A Brief Review
• Particles to Partial Differential Equations Parsimoniously
• Prospects of federated machine learning in fluid dynamics
• Hierarchical deep learning-based adaptive time-stepping scheme for multiscale simulations
• Reduced order modeling of dynamical systems using artificial neural networks applied to water circulation
• Data-Driven Forecasting of Non-Equilibrium Solid-State Dynamics
• Stability Preserving Data-driven Models With Latent Dynamics
• Critical Sampling for Robust Evolution Operator Learning of Unknown Dynamical Systems
• Reconstruction of observed mechanical motions with Artificial Intelligence tools
• A comparison of data-driven approaches to build low-dimensional ocean models