A spectral approach to Hebbian-like neural networks

Abstract

We consider the Hopfield neural network as a model of associative memory and we define its neuronal interaction matrix $\mathbf{J}$ as a function of a set of $K\times M$ binary vectors $\{{\xi }^{\mu ,A}{\}}_{\mu =1,...,K}^{A=1,...,M}$ representing a sample of the reality that we want to retrieve. In particular, any item ${\xi }^{\mu ,A}$ is meant as a corrupted version of an unknown ground pattern ${\zeta }^{\mu }$, that is the target of our retrieval process. We consider and compare two definitions for $\mathbf{J}$, referred to as supervised and unsupervised, according to whether the class $\mu$, each example belongs to, is unveiled or not, also, these definitions recover the paradigmatic Hebb's rule under suitable limits. The spectral properties of the resulting matrices are studied and used to inspect the retrieval capabilities of the related models as a function of their control parameters.

References in corpus (11)

• Statistical physics of inference: Thresholds and algorithms
• Universality of covariance matrices
• Cavity Approach to the Spectral Density of Sparse Symmetric Random Matrices
• Rational Decisions, Random Matrices and Spin Glasses
• Supervised Hebbian Learning
• How glassy are neural networks?
• Dreaming neural networks: rigorous results
• On the Critical Capacity of the Hopfield Model
• Eigenvalue spectrum of neural networks with arbitrary Hebbian length
• Unlearning regularization for Boltzmann Machines
• Interacting Dreaming Neural Networks