The Relativistic Hopfield network: rigorous results

Abstract

The relativistic Hopfield model constitutes a generalization of the standard Hopfield model that is derived by the formal analogy between the statistical-mechanic framework embedding neural networks and the Lagrangian mechanics describing a fictitious single-particle motion in the space of the tuneable parameters of the network itself. In this analogy the cost-function of the Hopfield model plays as the standard kinetic-energy term and its related Mattis overlap (naturally bounded by one) plays as the velocity. The Hamiltonian of the relativisitc model, once Taylor-expanded, results in a P-spin series with alternate signs: the attractive contributions enhance the information-storage capabilities of the network, while the repulsive contributions allow for an easier unlearning of spurious states, conferring overall more robustness to the system as a whole. Here we do not deepen the information processing skills of this generalized Hopfield network, rather we focus on its statistical mechanical foundation. In particular, relying on Guerra's interpolation techniques, we prove the existence of the infinite volume limit for the model free-energy and we give its explicit expression in terms of the Mattis overlaps. By extremizing the free energy over the latter we get the generalized self-consistent equations for these overlaps, as well as a picture of criticality that is further corroborated by a fluctuation analysis. These findings are in full agreement with the available previous results.

arXiv comment11 pages, 1 figure

References in corpus (12)

• The Thermodynamic Limit in Mean Field Spin Glass Models
• Multitasking associative networks
• Emergence of Compositional Representations in Restricted Boltzmann Machines
• Equilibrium statistical mechanics of bipartite spin systems
• The replica symmetric behavior of the analogical neural network
• Dense Associative Memory is Robust to Adversarial Inputs
• The mean field Ising model trough interpolating techniques
• From Dyson to Hopfield: Processing on hierarchical networks
• How glassy are neural networks?
• The infinite volume limit in generalized mean field disordered models
• Non-Convex Multi-species Hopfield models
• Neural Networks retrieving Boolean patterns in a sea of Gaussian ones

Cited by in corpus (3)

• Non-linear PDEs approach to statistical mechanics of Dense Associative Memories
• Explosive neural networks via higher-order interactions in curved statistical manifolds
• The relativistic Hopfield model with correlated patterns