Memory Uncertainty Relation and Harmonic Memory in Random Recurrent Networks

Abstract

We present an inequality that bounds the short-term memory capability of dynamical systems from below. It can be interpreted as an uncertainty relation between a measure of short-term memory and that of the size of state fluctuations induced by input signals. The lower bound can be achieved by a readout weight and thus represents a suboptimal memory called harmonic memory. We examine analytically and numerically the inequality in a number of reservoir systems subject to input noise. We illustrate cases in which equality is achieved exactly, equality holds asymptotically, and the inequality is strict. We also study the effect of a state-space regularization to elucidate the inequality in terms of the fluctuation structure of the state-space. We find that a certain strength of input noise induces extra memory under the regularization, and we refer to this phenomenon as noise-induced memory. We observe that the memory uncertainty relation does not hold in general for the regularized memory and harmonic memory. This fact is explained in terms of the mechanism of noise-induced memory.

14 pages, 10 figures

References in corpus (13)

• Recent Advances in Physical Reservoir Computing: A Review
• Thermodynamic uncertainty relation for biomolecular processes
• Dissipation bounds all steady-state current fluctuations
• Physical reservoir computing -- An introductory perspective
• Fluctuation Theorem with Information Exchange: Role of Correlations in Stochastic Thermodynamics
• Short-Term Memory in Orthogonal Neural Networks
• Role of Mutual Information in Entropy Production under Information Exchanges
• Optimal sequence memory in driven random networks
• Training Echo State Networks with Regularization through Dimensionality Reduction
• Mean Field Theory of Dynamical Systems Driven by External Signals
• The difference between memory and prediction in linear recurrent networks
• Optimal short-term memory before the edge of chaos in driven random recurrent networks
• Memory and forecasting capacities of nonlinear recurrent networks