STSC-SNN: Spatio-Temporal Synaptic Connection with Temporal Convolution and Attention for Spiking Neural Networks

Abstract

Spiking Neural Networks (SNNs), as one of the algorithmic models in neuromorphic computing, have gained a great deal of research attention owing to temporal information processing capability, low power consumption, and high biological plausibility. The potential to efficiently extract spatio-temporal features makes it suitable for processing the event streams. However, existing synaptic structures in SNNs are almost full-connections or spatial 2D convolution, neither of which can extract temporal dependencies adequately. In this work, we take inspiration from biological synapses and propose a spatio-temporal synaptic connection SNN (STSC-SNN) model, to enhance the spatio-temporal receptive fields of synaptic connections, thereby establishing temporal dependencies across layers. Concretely, we incorporate temporal convolution and attention mechanisms to implement synaptic filtering and gating functions. We show that endowing synaptic models with temporal dependencies can improve the performance of SNNs on classification tasks. In addition, we investigate the impact of performance vias varied spatial-temporal receptive fields and reevaluate the temporal modules in SNNs. Our approach is tested on neuromorphic datasets, including DVS128 Gesture (gesture recognition), N-MNIST, CIFAR10-DVS (image classification), and SHD (speech digit recognition). The results show that the proposed model outperforms the state-of-the-art accuracy on nearly all datasets.

References in corpus (12)

• Event-based Vision: A Survey
• Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks
• SLAYER: Spike Layer Error Reassignment in Time
• Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
• The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks
• Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting
• Direct Training for Spiking Neural Networks: Faster, Larger, Better
• TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networks
• Comparing SNNs and RNNs on Neuromorphic Vision Datasets: Similarities and Differences
• Going Deeper With Directly-Trained Larger Spiking Neural Networks
• Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network
• MAP-SNN: Mapping Spike Activities with Multiplicity, Adaptability, and Plasticity into Bio-Plausible Spiking Neural Networks

Cited by in corpus (7)

• Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings
• Loss shaping enhances exact gradient learning with Eventprop in spiking neural networks
• Precise spiking motifs in neurobiological and neuromorphic data
• Accelerating SNN Training with Stochastic Parallelizable Spiking Neurons
• Hardware-Accelerated Event-Graph Neural Networks for Low-Latency Time-Series Classification on SoC FPGA
• Accurate Detection of Spiking Motifs by Learning Heterogeneous Delays of a Spiking Neural Network
• Gated Parametric Neuron for Spike-based Audio Recognition