Kernel-based Prediction of Non-Markovian Time Series

Abstract

A nonparametric method to predict non-Markovian time series of partially observed dynamics is developed. The prediction problem we consider is a supervised learning task of finding a regression function that takes a delay embedded observable to the observable at a future time. When delay embedding theory is applicable, the proposed regression function is a consistent estimator of the flow map induced by the delay embedding. Furthermore, the corresponding Mori-Zwanzig equation governing the evolution of the observable simplifies to only a Markovian term, represented by the regression function. We realize this supervised learning task with a class of kernel-based linear estimators, the kernel analog forecast (KAF), which are consistent in the limit of large data. In a scenario with a high-dimensional covariate space, we employ a Markovian kernel smoothing method which is computationally cheaper than the Nyström projection method for realizing KAF. In addition to the guaranteed theoretical convergence, we numerically demonstrate the effectiveness of this approach on higher-dimensional problems where the relevant kernel features are difficult to capture with the Nyström method. Given noisy training data, we propose a nonparametric smoother as a de-noising method. Numerically, we show that the proposed smoother is more accurate than EnKF and 4Dvar in de-noising signals corrupted by independent (but not necessarily identically distributed) noise, even if the smoother is constructed using a data set corrupted by white noise. We show skillful prediction using the KAF constructed from the denoised data.

arXiv comment7 figures

References in corpus (14)

• Consistency of spectral clustering
• A Conceptual Introduction to Hamiltonian Monte Carlo
• Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks
• Data-driven Discovery of Closure Models
• Data-driven model reduction, Wiener projections, and the Koopman-Mori-Zwanzig formalism
• Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques
• Machine Learning for Prediction with Missing Dynamics
• A Priori Estimation Of Memory Effects In Coarse-Grained Nonlinear Systems Using The Mori-Zwanzig Formalism
• A Dynamic Subgrid Scale Model for Large Eddy Simulations Based on the Mori-Zwanzig Formalism
• Computation of the Memory Functions in the Generalized Langevin Models for Collective Dynamics of Macromolecules
• Data-driven prediction strategies for low-frequency patterns of North Pacific climate variability
• Predicting regional and pan-Arctic sea ice anomalies with kernel analog forecasting
• Parametric Reduced Models for the Nonlinear Schrödinger Equation
• Modeling of Missing Dynamical Systems: Deriving Parametric Models using a Nonparametric Framework

Cited by in corpus (12)

• Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations
• Theoretical tools for understanding the climate crisis from Hasselmann's program and beyond
• Learning Stochastic Dynamics with Statistics-Informed Neural Network
• A Markovian dynamics for C. elegans behavior across scales
• Maximally predictive states: from partial observations to long timescales
• Nonequilibrium Statistical Mechanics and Optimal Prediction of Partially-Observed Complex Systems
• Machine Learning-Based Statistical Closure Models for Turbulent Dynamical Systems
• A Data-Driven Statistical-Stochastic Surrogate Modeling Strategy for Complex Nonlinear Non-stationary Dynamics
• Memory-based reduced modelling and data-based estimation of opinion spreading
• Data-driven Mori-Zwanzig modeling of Lagrangian particle dynamics in turbulent flows
• Error Bounds of the Invariant Statistics in Machine Learning of Ergodic Itô Diffusions
• Manifold-based time series forecasting