How glassy are neural networks?

Abstract

In this paper we continue our investigation on the high storage regime of a neural network with Gaussian patterns. Through an exact mapping between its partition function and one of a bipartite spin glass (whose parties consist of Ising and Gaussian spins respectively), we give a complete control of the whole annealed region. The strategy explored is based on an interpolation between the bipartite system and two independent spin glasses built respectively by dichotomic and Gaussian spins: Critical line, behavior of the principal thermodynamic observables and their fluctuations as well as overlap fluctuations are obtained and discussed. Then, we move further, extending such an equivalence beyond the critical line, to explore the broken ergodicity phase under the assumption of replica symmetry and we show that the quenched free energy of this (analogical) Hopfield model can be described as a linear combination of the two quenched spin-glass free energies even in the replica symmetric framework.

References in corpus (6)

• Broken Replica Symmetry Bounds in the Mean Field Spin Glass Model
• About the overlap distribution in mean field spin glass models
• A thermodynamical perspective of immune capabilities
• Irreducible free energy expansion and overlaps locking in mean field spin glasses
• Mean-field cooperativity in chemical kinetics
• Notes on the polynomial identities in random overlap structures

Cited by in corpus (23)

• Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priors
• Phase transitions in Restricted Boltzmann Machines with generic priors
• Free energy and complexity of spherical bipartite models
• Replica Symmetry Breaking in Bipartite Spin Glasses and Neural Networks
• Non-Convex Multi-species Hopfield models
• Hierarchical neural networks perform both serial and parallel processing
• Mean field bipartite spin models treated with mechanical techniques
• Replica symmetry breaking in dense neural networks
• Dense Hebbian neural networks: a replica symmetric picture of supervised learning
• Meta-stable states in the hierarchical Dyson model drive parallel processing in the hierarchical Hopfield network
• Legendre Duality of Spherical and Gaussian Spin Glasses
• Hopfield model with planted patterns: a teacher-student self-supervised learning model
• Free energies of Boltzmann Machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit
• Dense Hopfield Networks in the Teacher-Student Setting
• The effect of priors on Learning with Restricted Boltzmann Machines
• PDE/statistical mechanics duality: relation between Guerra's interpolated $p$-spin ferromagnets and the Burgers hierarchy
• Modeling Structured Data Learning with Restricted Boltzmann Machines in the Teacher-Student Setting
• The Relativistic Hopfield network: rigorous results
• Dynamics of one-dimensional spin models under the line-graph operator
• The dual of the space of interactions in neural network models
• A spectral approach to Hebbian-like neural networks
• Supervised and Unsupervised protocols for hetero-associative neural networks
• Single-Nodal Spontaneous Symmetry Breaking in NLP Models