Supervised perceptron learning vs unsupervised Hebbian unlearning: Approaching optimal memory retrieval in Hopfield-like networks

Abstract

The Hebbian unlearning algorithm, i.e. an unsupervised local procedure used to improve the retrieval properties in Hopfield-like neural networks, is numerically compared to a supervised algorithm to train a linear symmetric perceptron. We analyze the stability of the stored memories: basins of attraction obtained by the Hebbian unlearning technique are found to be comparable in size to those obtained in the symmetric perceptron, while the two algorithms are found to converge in the same region of Gardner's space of interactions, having followed similar learning paths. A geometric interpretation of Hebbian unlearning is proposed to explain its optimal performances. Because the Hopfield model is also a prototypical model of disordered magnetic system, it might be possible to translate our results to other models of interest for memory storage in materials.

arXiv comment11 pages, 13 figures, 43 references

References in corpus (9)

• Memory formation in matter
• Directed aging, memory and Nature's greed
• Capacity-resolution trade-off in the optimal learning of multiple low-dimensional manifolds by attractor neural networks
• On the number of limit cycles in asymmetric neural networks
• On the number of limit cycles in diluted neural networks
• Neural Population Geometry Reveals the Role of Stochasticity in Robust Perception
• Dreams, endocannabinoids and itinerant dynamics in neural networks: re elaborating Crick-Mitchison unlearning hypothesis
• Recognition Capabilities of a Hopfield Model with Auxiliary Hidden Neurons
• Satisfiability transition in asymmetric neural networks

Cited by in corpus (8)

• Soft Metamaterials: Adaptation and Intelligence
• Matrix factorization with neural networks
• Daydreaming Hopfield Networks and their surprising effectiveness on correlated data
• Unlearning regularization for Boltzmann Machines
• Satisfiability transition in asymmetric neural networks
• Statistical mechanics of vector Hopfield network near and above saturation
• Pseudo-likelihood produces associative memories able to generalize, even for asymmetric couplings
• Daydreaming algorithm for Biased Patterns