Spiking Deep Residual Network

Abstract

Spiking neural networks (SNNs) have received significant attention for their biological plausibility. SNNs theoretically have at least the same computational power as traditional artificial neural networks (ANNs). They possess potential of achieving energy-efficiency while keeping comparable performance to deep neural networks (DNNs). However, it is still a big challenge to train a very deep SNN. In this paper, we propose an efficient approach to build a spiking version of deep residual network (ResNet). ResNet is considered as a kind of the state-of-the-art convolutional neural networks (CNNs). We employ the idea of converting a trained ResNet to a network of spiking neurons, named Spiking ResNet (S-ResNet). We propose a shortcut conversion model to appropriately scale continuous-valued activations to match firing rates in SNN, and a compensation mechanism to reduce the error caused by discretisation. Experimental results demonstrate that, compared with the state-of-the-art SNN approaches, the proposed Spiking ResNet achieves the best performance on CIFAR-10, CIFAR-100, and ImageNet 2012. Our work is the first time to build a SNN deeper than 40, with comparable performance to ANNs on a large-scale dataset.

References in corpus (8)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
• SLAYER: Spike Layer Error Reassignment in Time
• HFirst: A Temporal Approach to Object Recognition
• Spiking Deep Networks with LIF Neurons
• Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex
• Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines
• Direct Training for Spiking Neural Networks: Faster, Larger, Better

Cited by in corpus (15)

• Deep Residual Learning in Spiking Neural Networks
• A Biologically Plausible Supervised Learning Method for Spiking Neural Networks Using the Symmetric STDP Rule
• A Tandem Learning Rule for Effective Training and Rapid Inference of Deep Spiking Neural Networks
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks
• Optimal ANN-SNN Conversion for Fast and Accurate Inference in Deep Spiking Neural Networks
• Comprehensive SNN Compression Using ADMM Optimization and Activity Regularization
• Deep Spiking Neural Network with Spike Count based Learning Rule
• SpikePropamine: Differentiable Plasticity in Spiking Neural Networks
• Revealing Fine Structures of the Retinal Receptive Field by Deep Learning Networks
• Deep Spiking Neural Networks for Large Vocabulary Automatic Speech Recognition
• NatCSNN: A Convolutional Spiking Neural Network for recognition of objects extracted from natural images
• HYPER-SNN: Towards Energy-efficient Quantized Deep Spiking Neural Networks for Hyperspectral Image Classification
• Towards the Next Generation of Retinal Neuroprosthesis: Visual Computation with Spikes
• Spiking Neural Networks -- Part I: Detecting Spatial Patterns
• Shenjing: A low power reconfigurable neuromorphic accelerator with partial-sum and spike networks-on-chip