Catch-22s of reservoir computing

Abstract

Reservoir Computing (RC) is a simple and efficient model-free framework for forecasting the behavior of nonlinear dynamical systems from data. Here, we show that there exist commonly-studied systems for which leading RC frameworks struggle to learn the dynamics unless key information about the underlying system is already known. We focus on the important problem of basin prediction -- determining which attractor a system will converge to from its initial conditions. First, we show that the predictions of standard RC models (echo state networks) depend critically on warm-up time, requiring a warm-up trajectory containing almost the entire transient in order to identify the correct attractor. Accordingly, we turn to Next-Generation Reservoir Computing (NGRC), an attractive variant of RC that requires negligible warm-up time. By incorporating the exact nonlinearities in the original equations, we show that NGRC can accurately reconstruct intricate and high-dimensional basins of attraction, even with sparse training data (e.g., a single transient trajectory). Yet, a tiny uncertainty in the exact nonlinearity can render prediction accuracy no better than chance. Our results highlight the challenges faced by data-driven methods in learning the dynamics of multistable systems and suggest potential avenues to make these approaches more robust.

Published version (slight change to the title due to journal policy). Code at https://github.com/spcornelius/RCBasins

References in corpus (12)

• Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data
• Next Generation Reservoir Computing
• Forecasting Chaotic Systems with Very Low Connectivity Reservoir Computers
• A Bayesian machine scientist to aid in the solution of challenging scientific problems
• Do Reservoir Computers Work Best at the Edge of Chaos?
• Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations
• The Size of the Sync Basin Revisited
• Model-free prediction of multistability using echo state network
• Using Machine Learning to Anticipate Tipping Points and Extrapolate to Post-Tipping Dynamics of Non-Stationary Dynamical Systems
• Optimizing Memory in Reservoir Computers
• Learning unseen coexisting attractors
• Basins with tentacles

Cited by in corpus (5)

• Hybridizing Traditional and Next-Generation Reservoir Computing to Accurately and Efficiently Forecast Dynamical Systems
• How more data can hurt: Instability and regularization in next-generation reservoir computing
• Learning Beyond Experience: Generalizing to Unseen State Space with Reservoir Computing
• Tailored minimal reservoir computing: on the bidirectional connection between nonlinearities in the reservoir and in data
• Phase transitions from linear to nonlinear information processing in neural networks