A correlation game for unsupervised learning yields computational interpretations of Hebbian excitation, anti-Hebbian inhibition, and synapse elimination

Abstract

Much has been learned about plasticity of biological synapses from empirical studies. Hebbian plasticity is driven by correlated activity of presynaptic and postsynaptic neurons. Synapses that converge onto the same neuron often behave as if they compete for a fixed resource; some survive the competition while others are eliminated. To provide computational interpretations of these aspects of synaptic plasticity, we formulate unsupervised learning as a zero-sum game between Hebbian excitation and anti-Hebbian inhibition in a neural network model. The game formalizes the intuition that Hebbian excitation tries to maximize correlations of neurons with their inputs, while anti-Hebbian inhibition tries to decorrelate neurons from each other. We further include a model of synaptic competition, which enables a neuron to eliminate all connections except those from its most strongly correlated inputs. Through empirical studies, we show that this facilitates the learning of sensory features that resemble parts of objects.

References in corpus (2)

• A Hebbian/Anti-Hebbian Network Derived from Online Non-Negative Matrix Factorization Can Cluster and Discover Sparse Features
• A Hebbian/Anti-Hebbian Network for Online Sparse Dictionary Learning Derived from Symmetric Matrix Factorization

Cited by in corpus (7)

• Unsupervised Learning by Competing Hidden Units
• Weakly-correlated synapses promote dimension reduction in deep neural networks
• Local Unsupervised Learning for Image Analysis
• Dictionary Learning by Dynamical Neural Networks
• Neuroscience-inspired online unsupervised learning algorithms
• Unsupervised learning by a nonlinear network with Hebbian excitatory and anti-Hebbian inhibitory neurons
• Two "correlation games" for a nonlinear network with Hebbian excitatory neurons and anti-Hebbian inhibitory neurons