First-spike based visual categorization using reward-modulated STDP

Abstract

Reinforcement learning (RL) has recently regained popularity, with major achievements such as beating the European game of Go champion. Here, for the first time, we show that RL can be used efficiently to train a spiking neural network (SNN) to perform object recognition in natural images without using an external classifier. We used a feedforward convolutional SNN and a temporal coding scheme where the most strongly activated neurons fire first, while less activated ones fire later, or not at all. In the highest layers, each neuron was assigned to an object category, and it was assumed that the stimulus category was the category of the first neuron to fire. If this assumption was correct, the neuron was rewarded, i.e. spike-timing-dependent plasticity (STDP) was applied, which reinforced the neuron's selectivity. Otherwise, anti-STDP was applied, which encouraged the neuron to learn something else. As demonstrated on various image datasets (Caltech, ETH-80, and NORB), this reward modulated STDP (R-STDP) approach extracted particularly discriminative visual features, whereas classic unsupervised STDP extracts any feature that consistently repeats. As a result, R-STDP outperformed STDP on these datasets. Furthermore, R-STDP is suitable for online learning, and can adapt to drastic changes such as label permutations. Finally, it is worth mentioning that both feature extraction and classification were done with spikes, using at most one spike per neuron. Thus the network is hardware friendly and energy efficient.

supplementary materials are added, Caltech face/motorbike demonstration figure is updated, some parts of the main manuscript are moved to the supplementary materials, additional network analysis and performance comparison with deep nets are added

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
• STDP-based spiking deep convolutional neural networks for object recognition
• Bio-Inspired Spiking Convolutional Neural Network using Layer-wise Sparse Coding and STDP Learning
• Deep Spiking Networks
• Acquisition of Visual Features Through Probabilistic Spike-Timing-Dependent Plasticity
• STDP allows close-to-optimal spatiotemporal spike pattern detection by single coincidence detector neurons
• Humans and deep networks largely agree on which kinds of variation make object recognition harder
• A wake-sleep algorithm for recurrent, spiking neural networks
• Deep counter networks for asynchronous event-based processing

Cited by in corpus (20)

• Deep Learning in Spiking Neural Networks
• Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks
• Spiking Neural Networks Hardware Implementations and Challenges: a Survey
• BindsNET: A machine learning-oriented spiking neural networks library in Python
• S4NN: temporal backpropagation for spiking neural networks with one spike per neuron
• Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks
• Demonstrating Advantages of Neuromorphic Computation: A Pilot Study
• Ultra Low-Power and Real-time ECG Classification Based on STDP and R-STDP Neural Networks for Wearable Devices
• SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks with at most one Spike per Neuron
• BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning
• Constructing Accurate and Efficient Deep Spiking Neural Networks with Double-threshold and Augmented Schemes
• Unsupervised Visual Feature Learning with Spike-timing-dependent Plasticity: How Far are we from Traditional Feature Learning Approaches?
• Simple and complex spiking neurons: perspectives and analysis in a simple STDP scenario
• Spiking neural networks trained via proxy
• An optimised deep spiking neural network architecture without gradients
• A Tandem Learning Rule for Effective Training and Rapid Inference of Deep Spiking Neural Networks
• Connection Pruning for Deep Spiking Neural Networks with On-Chip Learning
• Efficient Implementation of a Multi-Layer Gradient-Free Online-Trainable Spiking Neural Network on FPGA
• Unsupervised End-to-End Training with a Self-Defined Target
• The backpropagation-based recollection hypothesis: Backpropagated action potentials mediate recall, imagination, language understanding and naming