Optimization theory of Hebbian/anti-Hebbian networks for PCA and whitening
arXiv:1511.09468 · doi:10.1109/ALLERTON.2015.7447180
Abstract
In analyzing information streamed by sensory organs, our brains face challenges similar to those solved in statistical signal processing. This suggests that biologically plausible implementations of online signal processing algorithms may model neural computation. Here, we focus on such workhorses of signal processing as Principal Component Analysis (PCA) and whitening which maximize information transmission in the presence of noise. We adopt the similarity matching framework, recently developed for principal subspace extraction, but modify the existing objective functions by adding a decorrelating term. From the modified objective functions, we derive online PCA and whitening algorithms which are implementable by neural networks with local learning rules, i.e. synaptic weight updates that depend on the activity of only pre- and postsynaptic neurons. Our theory offers a principled model of neural computations and makes testable predictions such as the dropout of underutilized neurons.
Annual Allerton Conference on Communication, Control, and Computing (Allerton) 2015
References in corpus (5)
- A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data
- A Hebbian/Anti-Hebbian Network Derived from Online Non-Negative Matrix Factorization Can Cluster and Discover Sparse Features
- A Normative Theory of Adaptive Dimensionality Reduction in Neural Networks
- A Hebbian/Anti-Hebbian Network for Online Sparse Dictionary Learning Derived from Symmetric Matrix Factorization
- A Neuron as a Signal Processing Device
Cited by in corpus (5)
- Blind nonnegative source separation using biological neural networks
- A Normative Theory of Adaptive Dimensionality Reduction in Neural Networks
- Hebbian Semi-Supervised Learning in a Sample Efficiency Setting
- Training Convolutional Neural Networks With Hebbian Principal Component Analysis
- Neuroscience-inspired online unsupervised learning algorithms