Phase Diagram and Storage Capacity of Sequence Processing Neural Networks

Abstract

We solve the dynamics of Hopfield-type neural networks which store sequences of patterns, close to saturation. The asymmetry of the interaction matrix in such models leads to violation of detailed balance, ruling out an equilibrium statistical mechanical analysis. Using generating functional methods we derive exact closed equations for dynamical order parameters, viz. the sequence overlap and correlation- and response functions, in the thermodynamic limit. We calculate the time translation invariant solutions of these equations, describing stationary limit-cycles, which leads to a phase diagram. The effective retarded self-interaction usually appearing in symmetric models is here found to vanish, which causes a significantly enlarged storage capacity of ${\alpha }_c\sim 0.269$, compared to $α_\c\sim 0.139$ for Hopfield networks storing static patterns. Our results are tested against extensive computer simulations and excellent agreement is found.

17 pages Latex2e, 2 postscript figures

Cited by in corpus (19)

• Parallel dynamics of disordered Ising spin systems on finitely connected random graphs
• Transient dynamics for sequence processing neural networks
• Rank abundance relations in evolutionary dynamics of random replicators
• The path-integral analysis of an associative memory model storing an infinite number of finite limit cycles
• Spectrum of non-Hermitian deep-Hebbian neural networks
• Generating functional analysis of CDMA detection dynamics
• Mean-field dynamics of sequence processing neural networks with finite connectivity
• Topology and Dynamics of Attractor Neural Networks: The Role of Loopiness
• Bifurcation analysis in an associative memory model
• Low-dimensional chaos induced by frustration in a non-monotonic system
• Period-two cycles in a feed-forward layered neural network model with symmetric sequence processing
• Theory of Recurrent Neural Network with Common Synaptic Inputs
• Transient dynamics for sequence processing neural networks: effect of degree distributions
• Pattern reconstruction and sequence processing in feed-forward layered neural networks near saturation
• The attractors in sequence processing neural networks
• Symmetric sequence processing in a recurrent neural network model with a synchronous dynamics
• Storage Capacity Diverges with Synaptic Efficiency in an Associative Memory Model with Synaptic Delay and Pruning
• Associative Memory by Recurrent Neural Networks with Delay Elements
• Statistical Neurodynamics for sequence processing neural networks with finite dilution