Forecasting Chaotic Systems with Very Low Connectivity Reservoir Computers

Abstract

We explore the hyperparameter space of reservoir computers used for forecasting of the chaotic Lorenz '63 attractor with Bayesian optimization. We use a new measure of reservoir performance, designed to emphasize learning the global climate of the forecasted system rather than short-term prediction. We find that optimizing over this measure more quickly excludes reservoirs that fail to reproduce the climate. The results of optimization are surprising: the optimized parameters often specify a reservoir network with very low connectivity. Inspired by this observation, we explore reservoir designs with even simpler structure, and find well-performing reservoirs that have zero spectral radius and no recurrence. These simple reservoirs provide counterexamples to widely used heuristics in the field, and may be useful for hardware implementations of reservoir computers.

References in corpus (5)

• Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data
• Good and bad predictions: Assessing and improving the replication of chaotic attractors by means of reservoir computing
• Rapid Time Series Prediction with a Hardware-Based Reservoir Computer
• Metric for attractor overlap
• Bayesian optimization of hyper-parameters in reservoir computing

Cited by in corpus (31)

• Next Generation Reservoir Computing
• On Explaining the Surprising Success of Reservoir Computing Forecaster of Chaos? The Universal Machine Learning Dynamical System with Contrasts to VAR and DMD
• Quantum reservoir computing with a single nonlinear oscillator
• Anticipating synchronization with machine learning
• Machine learning algorithms for predicting the amplitude of chaotic laser pulses
• Integrating Recurrent Neural Networks with Data Assimilation for Scalable Data-Driven State Estimation
• Using Machine Learning to Anticipate Tipping Points and Extrapolate to Post-Tipping Dynamics of Non-Stationary Dynamical Systems
• Breaking Symmetries of the Reservoir Equations in Echo State Networks
• Model-free prediction of multistability using echo state network
• Reservoir Computing Approach to Quantum State Measurement
• Adaptable Hamiltonian neural networks
• Learning Hamiltonian dynamics by reservoir computer
• Robust Forecasting using Predictive Generalized Synchronization in Reservoir Computing
• Finding nonlinear system equations and complex network structures from data: a sparse optimization approach
• Catch-22s of reservoir computing
• Reducing network size and improving prediction stability of reservoir computing
• Hybridizing Traditional and Next-Generation Reservoir Computing to Accurately and Efficiently Forecast Dynamical Systems
• Chaotic attractor reconstruction using small reservoirs -- the influence of topology
• Criticality in Reservoir Computer of Coupled Phase Oscillators
• Transfer learning of chaotic systems
• Cluster-based Input Weight Initialization for Echo State Networks
• Symmetry-Aware Reservoir Computing
• Predicting multi-parametric dynamics of externally forced oscillator using reservoir computing and minimal data
• Effect of temporal resolution on the reproduction of chaotic dynamics via reservoir computing
• Reservoir Computers with Random and Optimized Time-Shifts
• Learning unidirectional coupling using echo-state network
• Hyperparameter Optimization for Randomized Algorithms: A Case Study on Random Features
• Role of assortativity in predicting burst synchronization using echo state network
• Reduced-order adaptive synchronization in a chaotic neural network with parameter mismatch: A dynamical system vs. machine learning approach
• Practical Trainable Temporal Postprocessor for Multistate Quantum Measurement
• Prediction performance of random reservoirs with different topology for nonlinear dynamical systems with different number of degrees of freedom