Eigenvalue spectrum of neural networks with arbitrary Hebbian length
arXiv:2103.14324 · doi:10.1103/PhysRevE.104.064307
Abstract
Associative memory is a fundamental function in the brain. Here, we generalize the standard associative memory model to include long-range Hebbian interactions at the learning stage, corresponding to a large synaptic integration window. In our model, the Hebbian length can be arbitrarily large. The spectral density of the coupling matrix is derived using the replica method, which is also shown to be consistent with the results obtained by applying the free probability method. The maximal eigenvalue is then obtained by an iterative equation, related to the paramagnetic to spin glass transition in the model. Altogether, this work establishes the connection between the associative memory with arbitrary Hebbian length and the asymptotic eigen-spectrum of the neural-coupling matrix.
arXiv comment19 pages, 4 figures
References in corpus (12)
- Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rules
- Cleaning large correlation matrices: tools from random matrix theory
- Mean-field message-passing equations in the Hopfield model and its generalizations
- Law of addition in random matrix theory
- Mean-field theory for a spin-glass model of neural networks: TAP free energy and paramagnetic to spin-glass transition
- Cavity approach to the first eigenvalue problem in a family of symmetric random sparse matrices
- Statistical mechanics of unsupervised feature learning in a restricted Boltzmann machine with binary synapses
- Extended temporal association memory by inhibitory Hebbian learning
- Weakly-correlated synapses promote dimension reduction in deep neural networks
- Many-body approach to the dynamics of batch learning
- Associative memory model with arbitrary Hebbian length
- On the Spectrum of Multi-Space Euclidean Random Matrices