Neuroscience-inspired online unsupervised learning algorithms
arXiv:1908.01867
Abstract
Although the currently popular deep learning networks achieve unprecedented performance on some tasks, the human brain still has a monopoly on general intelligence. Motivated by this and biological implausibility of deep learning networks, we developed a family of biologically plausible artificial neural networks (NNs) for unsupervised learning. Our approach is based on optimizing principled objective functions containing a term that matches the pairwise similarity of outputs to the similarity of inputs, hence the name - similarity-based. Gradient-based online optimization of such similarity-based objective functions can be implemented by NNs with biologically plausible local learning rules. Similarity-based cost functions and associated NNs solve unsupervised learning tasks such as linear dimensionality reduction, sparse and/or nonnegative feature extraction, blind nonnegative source separation, clustering and manifold learning.
arXiv commentAccepted for publication in IEEE Signal Processing Magazine
References in corpus (8)
- Why do similarity matching objectives lead to Hebbian/anti-Hebbian networks?
- A Hebbian/Anti-Hebbian Network Derived from Online Non-Negative Matrix Factorization Can Cluster and Discover Sparse Features
- Blind nonnegative source separation using biological neural networks
- A Normative Theory of Adaptive Dimensionality Reduction in Neural Networks
- A Hebbian/Anti-Hebbian Network for Online Sparse Dictionary Learning Derived from Symmetric Matrix Factorization
- Optimization theory of Hebbian/anti-Hebbian networks for PCA and whitening
- A correlation game for unsupervised learning yields computational interpretations of Hebbian excitation, anti-Hebbian inhibition, and synapse elimination
- Clustering is semidefinitely not that hard: Nonnegative SDP for manifold disentangling