Machine Learning for Prediction with Missing Dynamics

Abstract

This article presents a general framework for recovering missing dynamical systems using available data and machine learning techniques. The proposed framework reformulates the prediction problem as a supervised learning problem to approximate a map that takes the memories of the resolved and identifiable unresolved variables to the missing components in the resolved dynamics. We demonstrate the effectiveness of the proposed framework with a theoretical guarantee of a path-wise convergence of the resolved variables up to finite time and numerical tests on prototypical models in various scientific domains. These include the 57-mode barotropic stress models with multiscale interactions that mimic the blocked and unblocked patterns observed in the atmosphere, the nonlinear Schrödinger equation which found many applications in physics such as optics and Bose-Einstein-Condense, the Kuramoto-Sivashinsky equation which spatiotemporal chaotic pattern formation models trapped ion mode in plasma and phase dynamics in reaction-diffusion systems. While many machine learning techniques can be used to validate the proposed framework, we found that recurrent neural networks outperform kernel regression methods in terms of recovering the trajectory of the resolved components and the equilibrium one-point and two-point statistics. This superb performance suggests that recurrent neural networks are an effective tool for recovering the missing dynamics that involves approximation of high-dimensional functions.

References in corpus (18)

• On the difficulty of training Recurrent Neural Networks
• Deep Network Approximation for Smooth Functions
• Data-driven non-Markovian closure models
• The Expressive Power of Neural Networks: A View from the Width
• Time-series learning of latent-space dynamics for reduced-order model closure
• On the Convergence Rate of Training Recurrent Neural Networks
• A Priori Estimates of the Population Risk for Two-layer Neural Networks
• A discrete approach to stochastic parametrization and dimensional reduction in nonlinear dynamics
• Data-driven Discovery of Closure Models
• Data-driven model reduction, Wiener projections, and the Koopman-Mori-Zwanzig formalism
• Adaptive Approximation and Generalization of Deep Neural Network with Intrinsic Dimensionality
• A Priori Estimation Of Memory Effects In Coarse-Grained Nonlinear Systems Using The Mori-Zwanzig Formalism
• A Dynamic Subgrid Scale Model for Large Eddy Simulations Based on the Mori-Zwanzig Formalism
• Comparison of continuous and discrete-time data-based modeling for hypoelliptic systems
• Linear theory for filtering nonlinear multiscale systems with model error
• On Generalization Bounds of a Family of Recurrent Neural Networks
• Parametric Reduced Models for the Nonlinear Schrödinger Equation
• Semiparametric forecasting and filtering: correcting low-dimensional model error in parametric models

Cited by in corpus (19)

• Data-Driven Variational Multiscale Reduced Order Models
• Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations
• Theoretical tools for understanding the climate crisis from Hasselmann's program and beyond
• Learning Stochastic Dynamics with Statistics-Informed Neural Network
• Kernel-based Prediction of Non-Markovian Time Series
• Discovery of interpretable structural model errors by combining Bayesian sparse regression and data assimilation: A chaotic Kuramoto-Sivashinsky test case
• Deep learning delay coordinate dynamics for chaotic attractors from partial observable data
• Data-driven construction of stochastic reduced dynamics encoded with non-Markovian features
• Data-driven model reduction for stochastic Burgers equations
• Machine Learning-Based Statistical Closure Models for Turbulent Dynamical Systems
• Learning dynamical systems from data: A simple cross-validation perspective, part III: Irregularly-Sampled Time Series
• A Data-Driven Statistical-Stochastic Surrogate Modeling Strategy for Complex Nonlinear Non-stationary Dynamics
• Shock trace prediction by reduced models for a viscous stochastic Burgers equation
• Limits of Learning Dynamical Systems
• The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation
• Data-driven discovery of quasiperiodically driven dynamics
• Multiscale and Nonlocal Learning for PDEs using Densely Connected RNNs
• Experimental Design for Missing Physics
• Linear Response Based Parameter Estimation in the Presence of Model Error