Meta-stable states in the hierarchical Dyson model drive parallel processing in the hierarchical Hopfield network

Abstract

In this paper we introduce and investigate the statistical mechanics of hierarchical neural networks: First, we approach these systems à la Mattis, by thinking at the Dyson model as a single-pattern hierarchical neural network and we discuss the stability of different retrievable states as predicted by the related self-consistencies obtained from a mean-field bound and from a bound that bypasses the mean-field limitation. The latter is worked out by properly reabsorbing fluctuations of the magnetization related to higher levels of the hierarchy into effective fields for the lower levels. Remarkably, mixing Amit's ansatz technique (to select candidate retrievable states) with the interpolation procedure (to solve for the free energy of these states) we prove that (due to gauge symmetry) the Dyson model accomplishes both serial and parallel processing. One step forward, we extend this scenario toward multiple stored patterns by implementing the Hebb prescription for learning within the couplings. This results in an Hopfield-like networks constrained on a hierarchical topology, for which, restricting to the low storage regime (where the number of patterns grows at most logarithmical with the amount of neurons), we prove the existence of the thermodynamic limit for the free energy and we give an explicit expression of its mean field bound and of the related improved bound

References in corpus (3)

• Broken Replica Symmetry Bounds in the Mean Field Spin Glass Model
• Overlap Interfaces in Hierarchical Spin-Glass models
• Analogue neural networks on correlated random graphs

Cited by in corpus (8)

• From Dyson to Hopfield: Processing on hierarchical networks
• Rényi entropies of the highly-excited states of multidimensional harmonic oscillators by use of strong Laguerre asymptotics
• Non-Convex Multi-species Hopfield models
• Hierarchical neural networks perform both serial and parallel processing
• First-passage phenomena in hierarchical networks
• Hopfield model with planted patterns: a teacher-student self-supervised learning model
• Topological properties of hierarchical networks
• Dense Hopfield Networks in the Teacher-Student Setting