Non-linear PDEs approach to statistical mechanics of Dense Associative Memories

Abstract

Dense associative memories (DAM), are widespread models in artificial intelligence used for pattern recognition tasks; computationally, they have been proven to be robust against adversarial input and theoretically, leveraging their analogy with spin-glass systems, they are usually treated by means of statistical-mechanics tools. Here we develop analytical methods, based on nonlinear PDEs, to investigate their functioning. In particular, we prove differential identities involving DAM partition function and macroscopic observables useful for a qualitative and quantitative analysis of the system. These results allow for a deeper comprehension of the mechanisms underlying DAMs and provide interdisciplinary tools for their study.

References in corpus (6)

• Deep Learning in Neural Networks: An Overview
• Broken Replica Symmetry Bounds in the Mean Field Spin Glass Model
• The mean field Ising model trough interpolating techniques
• Neural networks with redundant representation: detecting the undetectable
• On quantum and relativistic mechanical analogues in mean field spin models
• PDE/statistical mechanics duality: relation between Guerra's interpolated $p$-spin ferromagnets and the Burgers hierarchy

Cited by in corpus (6)

• Dense Hebbian neural networks: a replica symmetric picture of supervised learning
• Explosive neural networks via higher-order interactions in curved statistical manifolds
• Complete integrability and equilibrium thermodynamics of biaxial nematic systems with discrete orientational degrees of freedom
• Gauge theory for mixed $p$-spin glasses
• Hamilton-Jacobi equations from mean-field spin glasses
• On solutions to a novel non-evolutionary integrable 1+1 PDE