Time Series Forecasting Using Manifold Learning

Abstract

We address a three-tier numerical framework based on manifold learning for the forecasting of high-dimensional time series. At the first step, we embed the time series into a reduced low-dimensional space using a nonlinear manifold learning algorithm such as Locally Linear Embedding and Diffusion Maps. At the second step, we construct reduced-order regression models on the manifold, in particular Multivariate Autoregressive (MVAR) and Gaussian Process Regression (GPR) models, to forecast the embedded dynamics. At the final step, we lift the embedded time series back to the original high-dimensional space using Radial Basis Functions interpolation and Geometric Harmonics. For our illustrations, we test the forecasting performance of the proposed numerical scheme with four sets of time series: three synthetic stochastic ones resembling EEG signals produced from linear and nonlinear stochastic models with different model orders, and one real-world data set containing daily time series of 10 key foreign exchange rates (FOREX) spanning the time period 03/09/2001-29/10/2020. The forecasting performance of the proposed numerical scheme is assessed using the combinations of manifold learning, modelling and lifting approaches. We also provide a comparison with the Principal Component Analysis algorithm as well as with the naive random walk model and the MVAR and GPR models trained and implemented directly in the high-dimensional space.

References in corpus (19)

• Scikit-learn: Machine Learning in Python
• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• LSTM: A Search Space Odyssey
• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• A Data-Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
• Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks
• Molecular enhanced sampling with autoencoders: On-the-fly collective variable discovery and accelerated free energy landscape exploration
• The Gap-Tooth Method in Particle Simulations
• Coarse Brownian Dynamics for Nematic Liquid Crystals: Bifurcation Diagrams via Stochastic Simulation
• Data-driven model reduction, Wiener projections, and the Koopman-Mori-Zwanzig formalism
• Coarse-scale PDEs from fine-scale observations via machine learning
• Reduced models in chemical kinetics via nonlinear data-mining
• Reduced-space Gaussian Process Regression for Data-Driven Probabilistic Forecast of Chaotic Dynamical Systems
• On the Koopman operator of algorithms
• Time series forecasting with Gaussian Processes needs priors
• Can social microblogging be used to forecast intraday exchange rates?
• Predicting Spatio-Temporal Time Series Using Dimension Reduced Local States
• To Infinity and Beyond: Some ODE and PDE Case Studies
• When Locally Linear Embedding Hits Boundary

Cited by in corpus (5)

• Data-driven Control of Agent-based Models: an Equation/Variable-free Machine Learning Approach
• RandONet: Shallow-Networks with Random Projections for learning linear and nonlinear operators
• Data-driven modelling of brain activity using neural networks, Diffusion Maps, and the Koopman operator
• Slow Invariant Manifolds of Singularly Perturbed Systems via Physics-Informed Machine Learning
• Learning the Latent dynamics of Fluid flows from High-Fidelity Numerical Simulations using Parsimonious Diffusion Maps