Supervised Hebbian Learning

Abstract

In neural network's Literature, Hebbian learning traditionally refers to the procedure by which the Hopfield model and its generalizations store archetypes (i.e., definite patterns that are experienced just once to form the synaptic matrix). However, the term "Learning" in Machine Learning refers to the ability of the machine to extract features from the supplied dataset (e.g., made of blurred examples of these archetypes), in order to make its own representation of the unavailable archetypes. Here, given a sample of examples, we define a supervised learning protocol by which the Hopfield network can infer the archetypes, and we detect the correct control parameters (including size and quality of the dataset) to depict a phase diagram for the system performance. We also prove that, for structureless datasets, the Hopfield model equipped with this supervised learning rule is equivalent to a restricted Boltzmann machine and this suggests an optimal and interpretable training routine. Finally, this approach is generalized to structured datasets: we highlight a quasi-ultrametric organization (reminiscent of replica-symmetry-breaking) in the analyzed datasets and, consequently, we introduce an additional "replica hidden layer" for its (partial) disentanglement, which is shown to improve MNIST classification from 75% to 95%, and to offer a new perspective on deep architectures.

References in corpus (2)

• Mean-field message-passing equations in the Hopfield model and its generalizations
• Power-law Scaling to Assist with Key Challenges in Artificial Intelligence

Cited by in corpus (15)

• Artificial Intelligence-based algorithms in medical image scan seg-mentation and intelligent visual-content generation -- a concise overview
• Dense Hebbian neural networks: a replica symmetric picture of supervised learning
• Generalized hetero-associative neural networks
• Hopfield model with planted patterns: a teacher-student self-supervised learning model
• Eight challenges in developing theory of intelligence
• The unbearable lightness of Restricted Boltzmann Machines: Theoretical Insights and Biological Applications
• Parallel Learning by Multitasking Neural Networks
• Daydreaming Hopfield Networks and their surprising effectiveness on correlated data
• Non-linear classification capability of quantum neural networks due to emergent quantum metastability
• Synaptic plasticity alters the nature of chaos transition in neural networks
• A spectral approach to Hebbian-like neural networks
• Fermi-Bose Machine achieves both generalization and adversarial robustness
• Interacting Dreaming Neural Networks
• Supervised and Unsupervised protocols for hetero-associative neural networks
• Pseudo-likelihood produces associative memories able to generalize, even for asymmetric couplings