Vector Symbolic Finite State Machines in Attractor Neural Networks
arXiv:2212.01196 · doi:10.1162/neco_a_01638
Abstract
Hopfield attractor networks are robust distributed models of human memory, but lack a general mechanism for effecting state-dependent attractor transitions in response to input. We propose construction rules such that an attractor network may implement an arbitrary finite state machine (FSM), where states and stimuli are represented by high-dimensional random vectors, and all state transitions are enacted by the attractor network's dynamics. Numerical simulations show the capacity of the model, in terms of the maximum size of implementable FSM, to be linear in the size of the attractor network for dense bipolar state vectors, and approximately quadratic for sparse binary state vectors. We show that the model is robust to imprecise and noisy weights, and so a prime candidate for implementation with high-density but unreliable devices. By endowing attractor networks with the ability to emulate arbitrary FSMs, we propose a plausible path by which FSMs could exist as a distributed computational primitive in biological neural networks.
arXiv comment26 pages, 13 figures. This is the authors' final version before publication in Neural Computation
References in corpus (9)
- Weak pairwise correlations imply strongly correlated network states in a neural population
- Memory and information processing in neuromorphic systems
- A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations
- State dependent computation using coupled recurrent networks
- Metastable dynamics of neural circuits and networks
- Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving
- Learning may need only a few bits of synaptic precision
- Neural Distributed Autoassociative Memories: A Survey
- Toward the quantification of cognition