Deep learning algorithm for data-driven simulation of noisy dynamical system

Abstract

We present a deep learning model, DE-LSTM, for the simulation of a stochastic process with an underlying nonlinear dynamics. The deep learning model aims to approximate the probability density function of a stochastic process via numerical discretization and the underlying nonlinear dynamics is modeled by the Long Short-Term Memory (LSTM) network. It is shown that, when the numerical discretization is used, the function estimation problem can be solved by a multi-label classification problem. A penalized maximum log likelihood method is proposed to impose a smoothness condition in the prediction of the probability distribution. We show that the time evolution of the probability distribution can be computed by a high-dimensional integration of the transition probability of the LSTM internal states. A Monte Carlo algorithm to approximate the high-dimensional integration is outlined. The behavior of DE-LSTM is thoroughly investigated by using the Ornstein-Uhlenbeck process and noisy observations of nonlinear dynamical systems; Mackey-Glass time series and forced Van der Pol oscillator. It is shown that DE-LSTM makes a good prediction of the probability distribution without assuming any distributional properties of the stochastic process. For a multiple-step forecast of the Mackey-Glass time series, the prediction uncertainty, denoted by the 95\% confidence interval, first grows, then dynamically adjusts following the evolution of the system, while in the simulation of the forced Van der Pol oscillator, the prediction uncertainty does not grow in time even for a 3,000-step forecast.

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Deep Learning in Neural Networks: An Overview
• Auto-Encoding Variational Bayes
• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• A Recurrent Latent Variable Model for Sequential Data
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Data Based Identification and Prediction of Nonlinear and Complex Dynamical Systems
• Accelerating Eulerian Fluid Simulation With Convolutional Networks
• Learning Stochastic Recurrent Networks
• Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems
• Bayesian Recurrent Neural Networks
• Spatio-temporal phenomena in complex systems with time delays
• A Kalman filter powered by ${\mathcal{H}}^2$-matrices for quasi-continuous data assimilation problems

Cited by in corpus (25)

• On Explaining the Surprising Success of Reservoir Computing Forecaster of Chaos? The Universal Machine Learning Dynamical System with Contrasts to VAR and DMD
• A deep learning enabler for non-intrusive reduced order modeling of fluid flows
• Time-series learning of latent-space dynamics for reduced-order model closure
• Assessment of unsteady flow predictions using hybrid deep learning based reduced order models
• A non-intrusive reduced order modeling framework for quasi-geostrophic turbulence
• Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction
• A long short-term memory embedding for hybrid uplifted reduced order models
• Memory embedded non-intrusive reduced order modeling of non-ergodic flows
• Deep learning stochastic processes with QCD phase transition
• Coercing Machine Learning to Output Physically Accurate Results
• Data-driven Reconstruction of Nonlinear Dynamics from Sparse Observation
• Reinforcement learning for suppression of collective activity in oscillatory ensembles
• EM-like Learning Chaotic Dynamics from Noisy and Partial Observations
• Reinforcement Learning Framework for Deep Brain Stimulation Study
• Short note on the behavior of recurrent neural network for noisy dynamical system
• Incorporating Transformer and LSTM to Kalman Filter with EM algorithm for state estimation
• Differentiable programming and its applications to dynamical systems
• Variational Deep Learning for the Identification and Reconstruction of Chaotic and Stochastic Dynamical Systems from Noisy and Partial Observations
• Learning stochastic dynamical systems with neural networks mimicking the Euler-Maruyama scheme
• An energy-based deep splitting method for the nonlinear filtering problem
• Statistical Modeling for Spatio-Temporal Data from Stochastic Convection-Diffusion Processes
• Variational inference formulation for a model-free simulation of a dynamical system with unknown parameters by a recurrent neural network
• Data-Driven Continuum Dynamics via Transport-Teleport Duality
• Deep Chaos Synchronization
• Generative Adversarial Network for Probabilistic Forecast of Random Dynamical System