Reducing network size and improving prediction stability of reservoir computing

Abstract

Reservoir computing is a very promising approach for the prediction of complex nonlinear dynamical systems. Besides capturing the exact short-term trajectories of nonlinear systems, it has also proved to reproduce its characteristic long-term properties very accurately. However, predictions do not always work equivalently well. It has been shown that both short- and long-term predictions vary significantly among different random realizations of the reservoir. In order to gain an understanding on when reservoir computing works best, we investigate some differential properties of the respective realization of the reservoir in a systematic way. We find that removing nodes that correspond to the largest weights in the output regression matrix reduces outliers and improves overall prediction quality. Moreover, this allows to effectively reduce the network size and, therefore, increase computational efficiency. In addition, we use a nonlinear scaling factor in the hyperbolic tangent of the activation function. This adjusts the response of the activation function to the range of values of the input variables of the nodes. As a consequence, this reduces the number of outliers significantly and increases both the short- and long-term prediction quality for the nonlinear systems investigated in this study. Our results demonstrate that a large optimization potential lies in the systematical refinement of the differential reservoir properties for a given dataset.

arXiv comment11 pages, 8 figures, published in Chaos

References in corpus (12)

• Error and attack tolerance of complex networks
• Recent Advances in Physical Reservoir Computing: A Review
• Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data
• Attractor Reconstruction by Machine Learning
• Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model
• Forecasting Chaotic Systems with Very Low Connectivity Reservoir Computers
• Using a reservoir computer to learn chaotic attractors, with applications to chaos synchronisation and cryptography
• Using reservoir computers to distinguish chaotic signals
• Network Structure Effects in Reservoir Computers
• Good and bad predictions: Assessing and improving the replication of chaotic attractors by means of reservoir computing
• Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere
• A physics-aware machine to predict extreme events in turbulence

Cited by in corpus (7)

• Breaking Symmetries of the Reservoir Equations in Echo State Networks
• Nanowire networks: how does small-world character evolve with dimensionality?
• Predicting high-dimensional heterogeneous time series employing generalized local states
• Denoising and Reconstruction of Nonlinear Dynamics using Truncated Reservoir Computing
• Predicting three-dimensional chaotic systems with four qubit quantum systems
• Model-free inference of unseen attractors: Reconstructing phase space features from a single noisy trajectory using reservoir computing
• Node pruning reveals compact and optimal substructures within large networks