Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks

Abstract

Spiking neural networks (SNNs) are well suited for spatio-temporal learning and implementations on energy-efficient event-driven neuromorphic processors. However, existing SNN error backpropagation (BP) methods lack proper handling of spiking discontinuities and suffer from low performance compared with the BP methods for traditional artificial neural networks. In addition, a large number of time steps are typically required to achieve decent performance, leading to high latency and rendering spike-based computation unscalable to deep architectures. We present a novel Temporal Spike Sequence Learning Backpropagation (TSSL-BP) method for training deep SNNs, which breaks down error backpropagation across two types of inter-neuron and intra-neuron dependencies and leads to improved temporal learning precision. It captures inter-neuron dependencies through presynaptic firing times by considering the all-or-none characteristics of firing activities and captures intra-neuron dependencies by handling the internal evolution of each neuronal state in time. TSSL-BP efficiently trains deep SNNs within a much shortened temporal window of a few steps while improving the accuracy for various image classification datasets including CIFAR10.

arXiv commentAccepted for spotlight presentation of NeurIPS (Neural Information Processing System) 2020: https://proceedings.neurips.cc/paper/2020/hash/8bdb5058376143fa358981954e7626b8-Abstract.html

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Deep Learning in Spiking Neural Networks
• SLAYER: Spike Layer Error Reassignment in Time
• Training Spiking Deep Networks for Neuromorphic Hardware
• Direct Training for Spiking Neural Networks: Faster, Larger, Better
• Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks
• Spike-Train Level Backpropagation for Training Deep Recurrent Spiking Neural Networks

Cited by in corpus (7)

• DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networks
• BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning
• SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural Networks
• Automotive Object Detection via Learning Sparse Events by Spiking Neurons
• STCSNN: High energy efficiency spike-train level spiking neural networks with spatio-temporal conversion
• Attacking the Spike: On the Transferability and Security of Spiking Neural Networks to Adversarial Examples
• Skip-Connected Self-Recurrent Spiking Neural Networks with Joint Intrinsic Parameter and Synaptic Weight Training