Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks

Abstract

Spiking neural networks (SNNs) have shown clear advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency, due to their event-driven nature and sparse communication. However, the training of deep SNNs is not straightforward. In this paper, we propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition, which is referred to as progressive tandem learning of deep SNNs. By studying the equivalence between ANNs and SNNs in the discrete representation space, a primitive network conversion method is introduced that takes full advantage of spike count to approximate the activation value of analog neurons. To compensate for the approximation errors arising from the primitive network conversion, we further introduce a layer-wise learning method with an adaptive training scheduler to fine-tune the network weights. The progressive tandem learning framework also allows hardware constraints, such as limited weight precision and fan-in connections, to be progressively imposed during training. The SNNs thus trained have demonstrated remarkable classification and regression capabilities on large-scale object recognition, image reconstruction, and speech separation tasks, while requiring at least an order of magnitude reduced inference time and synaptic operations than other state-of-the-art SNN implementations. It, therefore, opens up a myriad of opportunities for pervasive mobile and embedded devices with a limited power budget.

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• WaveNet: A Generative Model for Raw Audio
• Neural Discrete Representation Learning
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks
• SLAYER: Spike Layer Error Reassignment in Time
• Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures
• Surrogate Gradient Learning in Spiking Neural Networks
• Training Spiking Deep Networks for Neuromorphic Hardware
• Direct Training for Spiking Neural Networks: Faster, Larger, Better
• Whetstone: A Method for Training Deep Artificial Neural Networks for Binary Communication
• Spiking Deep Residual Network
• Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection
• A Tandem Learning Rule for Effective Training and Rapid Inference of Deep Spiking Neural Networks