Stochastic Reservoir Computers

Abstract

Reservoir computing is a form of machine learning that utilizes nonlinear dynamical systems to perform complex tasks in a cost-effective manner when compared to typical neural networks. Many recent advancements in reservoir computing, in particular quantum reservoir computing, make use of reservoirs that are inherently stochastic. However, the theoretical justification for using these systems has not yet been well established. In this paper, we investigate the universality of stochastic reservoir computers, in which we use a stochastic system for reservoir computing using the probabilities of each reservoir state as the readout instead of the states themselves. In stochastic reservoir computing, the number of distinct states of the entire reservoir computer can potentially scale exponentially with the size of the reservoir hardware, offering the advantage of compact device size. We prove that classes of stochastic echo state networks, and therefore the class of all stochastic reservoir computers, are universal approximating classes. We also investigate the performance of two practical examples of stochastic reservoir computers in classification and chaotic time series prediction. While shot noise is a limiting factor in the performance of stochastic reservoir computing, we show significantly improved performance compared to a deterministic reservoir computer with similar hardware in cases where the effects of noise are small.

34 pages, 8 figures

References in corpus (18)

• Barren plateaus in quantum neural network training landscapes
• Recent Advances in Physical Reservoir Computing: A Review
• Large-Scale Optical Reservoir Computing for Spatiotemporal Chaotic Systems Prediction
• A full-stack view of probabilistic computing with p-bits: devices, architectures and algorithms
• Embedding and Approximation Theorems for Echo State Networks
• Using a reservoir computer to learn chaotic attractors, with applications to chaos synchronisation and cryptography
• Time Series Quantum Reservoir Computing with Weak and Projective Measurements
• Gaussian states of continuous-variable quantum systems provide universal and versatile reservoir computing
• Rapid Time Series Prediction with a Hardware-Based Reservoir Computer
• Large-scale spatiotemporal photonic reservoir computer for image classification
• Biasing the quantum vacuum to control macroscopic probability distributions
• Learning Nonlinear Input-Output Maps with Dissipative Quantum Systems
• Quantum reservoir computing in finite dimensions
• Exploring quantum mechanical advantage for reservoir computing
• Quantum-limited stochastic optical neural networks operating at a few quanta per activation
• Tackling Sampling Noise in Physical Systems for Machine Learning Applications: Fundamental Limits and Eigentasks
• Universality and approximation bounds for echo state networks with random weights
• Universal Time-Uniform Trajectory Approximation for Random Dynamical Systems with Recurrent Neural Networks

Cited by in corpus (2)

• Model-free Forecasting of Rogue Waves using Reservoir Computing
• Restrictions on Physical Stochastic Reservoir Computers