Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networks

Abstract

Spiking neural networks (SNNs) are biology-inspired artificial neural networks (ANNs) that comprise of spiking neurons to process asynchronous discrete signals. While more efficient in power consumption and inference speed on the neuromorphic hardware, SNNs are usually difficult to train directly from scratch with spikes due to the discreteness. As an alternative, many efforts have been devoted to converting conventional ANNs into SNNs by copying the weights from ANNs and adjusting the spiking threshold potential of neurons in SNNs. Researchers have designed new SNN architectures and conversion algorithms to diminish the conversion error. However, an effective conversion should address the difference between the SNN and ANN architectures with an efficient approximation \DSK{of} the loss function, which is missing in the field. In this work, we analyze the conversion error by recursive reduction to layer-wise summation and propose a novel strategic pipeline that transfers the weights to the target SNN by combining threshold balance and soft-reset mechanisms. This pipeline enables almost no accuracy loss between the converted SNNs and conventional ANNs with only $\sim 1/10$ of the typical SNN simulation time. Our method is promising to get implanted onto embedded platforms with better support of SNNs with limited energy and memory.

References in corpus (3)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Theory and Tools for the Conversion of Analog to Spiking Convolutional Neural Networks
• Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation

Cited by in corpus (9)

• Deep Residual Learning in Spiking Neural Networks
• Keys to Accurate Feature Extraction Using Residual Spiking Neural Networks
• Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings
• A Little Energy Goes a Long Way: Build an Energy-Efficient, Accurate Spiking Neural Network from Convolutional Neural Network
• SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural Networks
• Fractional-order spike-timing-dependent gradient descent for multi-layer spiking neural networks
• Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State
• One Timestep is All You Need: Training Spiking Neural Networks with Ultra Low Latency
• SPENSER: Towards a NeuroEvolutionary Approach for Convolutional Spiking Neural Networks