CSNNs: Unsupervised, Backpropagation-free Convolutional Neural Networks for Representation Learning
arXiv:2001.10388 · doi:10.1109/ICMLA.2019.00265
Abstract
This work combines Convolutional Neural Networks (CNNs), clustering via Self-Organizing Maps (SOMs) and Hebbian Learning to propose the building blocks of Convolutional Self-Organizing Neural Networks (CSNNs), which learn representations in an unsupervised and Backpropagation-free manner. Our approach replaces the learning of traditional convolutional layers from CNNs with the competitive learning procedure of SOMs and simultaneously learns local masks between those layers with separate Hebbian-like learning rules to overcome the problem of disentangling factors of variation when filters are learned through clustering. We investigate the learned representation by designing two simple models with our building blocks, achieving comparable performance to many methods which use Backpropagation, while we reach comparable performance on Cifar10 and give baseline performances on Cifar100, Tiny ImageNet and a small subset of ImageNet for Backpropagation-free methods.
18 pages,18 figures, Author's extended version of the paper. Final version presented at 18th IEEE International Conference on Machine Learning and Applications (ICMLA). Boca Raton, Florida / USA. 2019
References in corpus (13)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Learning deep representations by mutual information estimation and maximization
- Unsupervised Representation Learning by Predicting Image Rotations
- Large Scale Adversarial Representation Learning
- Recent Advances in Autoencoder-Based Representation Learning
- Towards Biologically Plausible Deep Learning
- Unsupervised Learning by Competing Hidden Units
- Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
- Convolutional Clustering for Unsupervised Learning
- Revisiting Self-Supervised Visual Representation Learning
- Unsupervised Discovery of Parts, Structure, and Dynamics
- Sparsely Activated Networks
- Evolving Losses for Unlabeled Video Representation Learning