Risk bounds for reservoir computing

Abstract

We analyze the practices of reservoir computing in the framework of statistical learning theory. In particular, we derive finite sample upper bounds for the generalization error committed by specific families of reservoir computing systems when processing discrete-time inputs under various hypotheses on their dependence structure. Non-asymptotic bounds are explicitly written down in terms of the multivariate Rademacher complexities of the reservoir systems and the weak dependence structure of the signals that are being handled. This allows, in particular, to determine the minimal number of observations needed in order to guarantee a prescribed estimation accuracy with high probability for a given reservoir family. At the same time, the asymptotic behavior of the devised bounds guarantees the consistency of the empirical risk minimization procedure for various hypothesis classes of reservoir functionals.

60 pages

References in corpus (8)

• Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data
• Attractor Reconstruction by Machine Learning
• High performance photonic reservoir computer based on a coherently driven passive cavity
• Short-Term Memory in Orthogonal Neural Networks
• The difference between memory and prediction in linear recurrent networks
• Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere
• The Asymptotic Performance of Linear Echo State Neural Networks
• Theory and Algorithms for Forecasting Time Series

Cited by in corpus (6)

• Chaos on compact manifolds: Differentiable synchronizations beyond the Takens theorem
• Memory and forecasting capacities of nonlinear recurrent networks
• Reservoir Computing for Macroeconomic Forecasting with Mixed Frequency Data
• Transport in reservoir computing
• Universality and approximation bounds for echo state networks with random weights
• State-space systems as dynamic generative models