Meta-stable states in the hierarchical Dyson model drive parallel processing in the hierarchical Hopfield network
arXiv:1407.5176 · doi:10.1088/1751-8113/48/1/015001
Abstract
In this paper we introduce and investigate the statistical mechanics of hierarchical neural networks: First, we approach these systems à la Mattis, by thinking at the Dyson model as a single-pattern hierarchical neural network and we discuss the stability of different retrievable states as predicted by the related self-consistencies obtained from a mean-field bound and from a bound that bypasses the mean-field limitation. The latter is worked out by properly reabsorbing fluctuations of the magnetization related to higher levels of the hierarchy into effective fields for the lower levels. Remarkably, mixing Amit's ansatz technique (to select candidate retrievable states) with the interpolation procedure (to solve for the free energy of these states) we prove that (due to gauge symmetry) the Dyson model accomplishes both serial and parallel processing. One step forward, we extend this scenario toward multiple stored patterns by implementing the Hebb prescription for learning within the couplings. This results in an Hopfield-like networks constrained on a hierarchical topology, for which, restricting to the low storage regime (where the number of patterns grows at most logarithmical with the amount of neurons), we prove the existence of the thermodynamic limit for the free energy and we give an explicit expression of its mean field bound and of the related improved bound
References in corpus (23)
- Mean-field theory for scale-free random networks
- Broken Replica Symmetry Bounds in the Mean Field Spin Glass Model
- Griffiths phases and the stretching of criticality in brain networks
- The Thermodynamic Limit in Mean Field Spin Glass Models
- Multitasking associative networks
- Relations between Short Range and Long Range Ising models
- The replica symmetric behavior of the analogical neural network
- Finite Connectivity Attractor Neural Networks
- The Hierarchical Random Energy Model
- Ensemble renormalization group for disordered systems
- How glassy are neural networks?
- Analytic solution of attractor neural networks on scale-free graphs
- Real space Renormalization Group analysis of a non-mean field spin-glass
- Overlap Interfaces in Hierarchical Spin-Glass models
- Parallel processing in immune networks
- Transition between localized and extended states in the hierarchical Anderson model
- Scaling of the largest dynamical barrier in the one-dimensional long-range Ising spin-glass
- Ensemble renormalization group for the random field hierarchical model
- Free-energy bounds for hierarchical spin models
- Equilibrium statistical mechanics on correlated random graphs
- Analogue neural networks on correlated random graphs
- Critical Exponents of the Random Field Hierarchical Model
- The renormalization flow of the hierarchical Anderson model at weak disorder
Cited by in corpus (8)
- From Dyson to Hopfield: Processing on hierarchical networks
- Rényi entropies of the highly-excited states of multidimensional harmonic oscillators by use of strong Laguerre asymptotics
- Non-Convex Multi-species Hopfield models
- Hierarchical neural networks perform both serial and parallel processing
- First-passage phenomena in hierarchical networks
- Hopfield model with planted patterns: a teacher-student self-supervised learning model
- Topological properties of hierarchical networks
- Dense Hopfield Networks in the Teacher-Student Setting