Improving STDP-based Visual Feature Learning with Whitening

Abstract

In recent years, spiking neural networks (SNNs) emerge as an alternative to deep neural networks (DNNs). SNNs present a higher computational efficiency using low-power neuromorphic hardware and require less labeled data for training using local and unsupervised learning rules such as spike timing-dependent plasticity (STDP). SNN have proven their effectiveness in image classification on simple datasets such as MNIST. However, to process natural images, a pre-processing step is required. Difference-of-Gaussians (DoG) filtering is typically used together with on-center/off-center coding, but it results in a loss of information that is detrimental to the classification performance. In this paper, we propose to use whitening as a pre-processing step before learning features with STDP. Experiments on CIFAR-10 show that whitening allows STDP to learn visual features that are closer to the ones learned with standard neural networks, with a significantly increased classification performance as compared to DoG filtering. We also propose an approximation of whitening as convolution kernels that is computationally cheaper to learn and more suited to be implemented on neuromorphic hardware. Experiments on CIFAR-10 show that it performs similarly to regular whitening. Cross-dataset experiments on CIFAR-10 and STL-10 also show that it is fairly stable across datasets, making it possible to learn a single whitening transformation to process different datasets.

References in corpus (7)

• Deep Learning in Spiking Neural Networks
• STDP-based spiking deep convolutional neural networks for object recognition
• A Survey of Neuromorphic Computing and Neural Networks in Hardware
• Bio-inspired Unsupervised Learning of Visual Features Leads to Robust Invariant Object Recognition
• Bio-Inspired Spiking Convolutional Neural Network using Layer-wise Sparse Coding and STDP Learning
• Decorrelated Batch Normalization
• Unsupervised Visual Feature Learning with Spike-timing-dependent Plasticity: How Far are we from Traditional Feature Learning Approaches?

Cited by in corpus (1)

• Ultra-low power on-chip learning of speech commands with phase-change memories