Non-linear PDEs approach to statistical mechanics of Dense Associative Memories

Abstract

Dense associative memories (DAM), are widespread models in artificial intelligence used for pattern recognition tasks; computationally, they have been proven to be robust against adversarial input and theoretically, leveraging their analogy with spin-glass systems, they are usually treated by means of statistical-mechanics tools. Here we develop analytical methods, based on nonlinear PDEs, to investigate their functioning. In particular, we prove differential identities involving DAM partition function and macroscopic observables useful for a qualitative and quantitative analysis of the system. These results allow for a deeper comprehension of the mechanisms underlying DAMs and provide interdisciplinary tools for their study.

References in corpus (21)

• Deep Learning in Neural Networks: An Overview
• Broken Replica Symmetry Bounds in the Mean Field Spin Glass Model
• General properties of overlap probability distributions in disordered spin systems. Toward Parisi ultrametricity
• Dense Associative Memory is Robust to Adversarial Inputs
• The mean field Ising model trough interpolating techniques
• A connection between the Ghirlanda--Guerra identities and ultrametricity
• Replica Symmetry Breaking in Attractor Neural Network Models
• Replica symmetry breaking in mean field spin glasses trough Hamilton-Jacobi technique
• Neural networks with redundant representation: detecting the undetectable
• Universality in bipartite mean field spin glasses
• Non-Convex Multi-species Hopfield models
• Dense Associative Memory for Pattern Recognition
• Shock dynamics of phase diagrams
• Notes on the p-spin glass studied via Hamilton-Jacobi and Smooth-Cavity techniques
• On quantum and relativistic mechanical analogues in mean field spin models
• Thermodynamic phase transitions and shock singularities
• Integrable extended van der Waals model
• Free energies of Boltzmann Machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit
• An exact study of phase transitions in mean field Potts models
• PDE/statistical mechanics duality: relation between Guerra's interpolated $p$-spin ferromagnets and the Burgers hierarchy
• The Relativistic Hopfield network: rigorous results

Cited by in corpus (6)

• Dense Hebbian neural networks: a replica symmetric picture of supervised learning
• Explosive neural networks via higher-order interactions in curved statistical manifolds
• Complete integrability and equilibrium thermodynamics of biaxial nematic systems with discrete orientational degrees of freedom
• Gauge theory for mixed $p$-spin glasses
• Hamilton-Jacobi equations from mean-field spin glasses
• On solutions to a novel non-evolutionary integrable 1+1 PDE