Bio-Inspired Spiking Convolutional Neural Network using Layer-wise Sparse Coding and STDP Learning

Abstract

Hierarchical feature discovery using non-spiking convolutional neural networks (CNNs) has attracted much recent interest in machine learning and computer vision. However, it is still not well understood how to create a biologically plausible network of brain-like, spiking neurons with multi-layer, unsupervised learning. This paper explores a novel bio-inspired spiking CNN that is trained in a greedy, layer-wise fashion. The proposed network consists of a spiking convolutional-pooling layer followed by a feature discovery layer extracting independent visual features. Kernels for the convolutional layer are trained using local learning. The learning is implemented using a sparse, spiking auto-encoder representing primary visual features. The feature discovery layer extracts independent features by probabilistic, leaky integrate-and-fire (LIF) neurons that are sparsely active in response to stimuli. The layer of the probabilistic, LIF neurons implicitly provides lateral inhibitions to extract sparse and independent features. Experimental results show that the convolutional layer is stack-admissible, enabling it to support a multi-layer learning. The visual features obtained from the proposed probabilistic LIF neurons in the feature discovery layer are utilized for training a classifier. Classification results contribute to the independent and informative visual features extracted in a hierarchy of convolutional and feature discovery layers. The proposed model is evaluated on the MNIST digit dataset using clean and noisy images. The recognition performance for clean images is above 98%. The performance loss for recognizing the noisy images is in the range 0.1% to 8.5% depending on noise types and densities. This level of performance loss indicates that the network is robust to additive noise.

References in corpus (4)

• Deep Learning in Neural Networks: An Overview
• STDP-based spiking deep convolutional neural networks for object recognition
• A Spiking Network that Learns to Extract Spike Signatures from Speech Signals
• Acquisition of Visual Features Through Probabilistic Spike-Timing-Dependent Plasticity

Cited by in corpus (17)

• Deep Learning in Spiking Neural Networks
• Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks
• Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures
• First-spike based visual categorization using reward-modulated STDP
• Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks
• SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks with at most one Spike per Neuron
• Biologically plausible deep learning -- but how far can we go with shallow networks?
• Representation Learning using Event-based STDP
• Dendritic-Inspired Processing Enables Bio-Plausible STDP in Compound Binary Synapses
• Unsupervised Clustering of Time Series Signals using Neuromorphic Energy-Efficient Temporal Neural Networks
• Improving STDP-based Visual Feature Learning with Whitening
• EventNet: Asynchronous Recursive Event Processing
• Training Deep Spiking Auto-encoders without Bursting or Dying Neurons through Regularization
• A Fully Spiking Hybrid Neural Network for Energy-Efficient Object Detection
• Spiking Two-Stream Methods with Unsupervised STDP-based Learning for Action Recognition
• Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation
• ScieNet: Deep Learning with Spike-assisted Contextual Information Extraction