Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks

Abstract

Compared with artificial neural networks (ANNs), spiking neural networks (SNNs) are promising to explore the brain-like behaviors since the spikes could encode more spatio-temporal information. Although pre-training from ANN or direct training based on backpropagation (BP) makes the supervised training of SNNs possible, these methods only exploit the networks' spatial domain information which leads to the performance bottleneck and requires many complicated training skills. Another fundamental issue is that the spike activity is naturally non-differentiable which causes great difficulties in training SNNs. To this end, we build an iterative LIF model that is more friendly for gradient descent training. By simultaneously considering the layer-by-layer spatial domain (SD) and the timing-dependent temporal domain (TD) in the training phase, as well as an approximated derivative for the spike activity, we propose a spatio-temporal backpropagation (STBP) training framework without using any complicated technology. We achieve the best performance of multi-layered perceptron (MLP) compared with existing state-of-the-art algorithms over the static MNIST and the dynamic N-MNIST dataset as well as a custom object detection dataset. This work provides a new perspective to explore the high-performance SNNs for future brain-like computing paradigm with rich spatio-temporal dynamics.

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
• Gated Feedback Recurrent Neural Networks
• Event-Driven Contrastive Divergence for Spiking Neuromorphic Systems
• Spiking Deep Networks with LIF Neurons
• Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences
• First-spike based visual categorization using reward-modulated STDP
• Bio-inspired Unsupervised Learning of Visual Features Leads to Robust Invariant Object Recognition
• Bio-Inspired Spiking Convolutional Neural Network using Layer-wise Sparse Coding and STDP Learning
• Deep Spiking Networks

Cited by in corpus (115)

• Event-based Vision: A Survey
• Deep Learning in Spiking Neural Networks
• Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures
• Spiking Neural Networks Hardware Implementations and Challenges: a Survey
• BindsNET: A machine learning-oriented spiking neural networks library in Python
• Deep Residual Learning in Spiking Neural Networks
• Unsupervised Learning of a Hierarchical Spiking Neural Network for Optical Flow Estimation: From Events to Global Motion Perception
• A Biologically Plausible Supervised Learning Method for Spiking Neural Networks Using the Symmetric STDP Rule
• Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks
• Deep learning incorporating biologically-inspired neural dynamics
• Speck: A Smart event-based Vision Sensor with a low latency 327K Neuron Convolutional Neuronal Network Processing Pipeline
• SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks with at most one Spike per Neuron
• Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networks
• An Introduction to Probabilistic Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications
• DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networks
• Federated Learning with Spiking Neural Networks
• Is Neuromorphic MNIST neuromorphic? Analyzing the discriminative power of neuromorphic datasets in the time domain
• Photonic spiking neural networks and CMOS-compatible graphene-on-silicon spiking neurons
• Constructing Accurate and Efficient Deep Spiking Neural Networks with Double-threshold and Augmented Schemes
• STSC-SNN: Spatio-Temporal Synaptic Connection with Temporal Convolution and Attention for Spiking Neural Networks
• TCL: an ANN-to-SNN Conversion with Trainable Clipping Layers
• Pruning of Deep Spiking Neural Networks through Gradient Rewiring
• Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training
• Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks
• Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation
• The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware
• Deep Reinforcement Learning with Population-Coded Spiking Neural Network for Continuous Control
• RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network
• Towards Lossless ANN-SNN Conversion under Ultra-Low Latency with Dual-Phase Optimization
• Keys to Accurate Feature Extraction Using Residual Spiking Neural Networks
• Exploiting Noise as a Resource for Computation and Learning in Spiking Neural Networks
• Encrypted Internet traffic classification using a supervised Spiking Neural Network
• DashNet: A Hybrid Artificial and Spiking Neural Network for High-speed Object Tracking
• An Energy-Efficient Spiking Neural Network for Finger Velocity Decoding for Implantable Brain-Machine Interface
• Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars
• Artificial Intelligence-based algorithms in medical image scan seg-mentation and intelligent visual-content generation -- a concise overview
• Integration of Leaky-Integrate-and-Fire-Neurons in Deep Learning Architectures
• In the Blink of an Eye: Event-based Emotion Recognition
• Continuous learning of spiking networks trained with local rules
• Reliable Object Tracking by Multimodal Hybrid Feature Extraction and Transformer-Based Fusion
• Loss shaping enhances exact gradient learning with Eventprop in spiking neural networks
• Empirical study on the efficiency of Spiking Neural Networks with axonal delays, and algorithm-hardware benchmarking
• SNN4Agents: A Framework for Developing Energy-Efficient Embodied Spiking Neural Networks for Autonomous Agents
• Exploring Adversarial Attack in Spiking Neural Networks with Spike-Compatible Gradient
• Comparing SNNs and RNNs on Neuromorphic Vision Datasets: Similarities and Differences
• N-Omniglot, a large-scale neuromorphic dataset for spatio-temporal sparse few-shot learning
• Event Transformer+. A multi-purpose solution for efficient event data processing
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks
• Evolutionary Spiking Neural Networks: A Survey
• SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with Spikes
• SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural Networks
• Reinforcement co-Learning of Deep and Spiking Neural Networks for Energy-Efficient Mapless Navigation with Neuromorphic Hardware
• Comprehensive SNN Compression Using ADMM Optimization and Activity Regularization
• Towards Low-Latency Energy-Efficient Deep SNNs via Attention-Guided Compression
• Temporal State Machines: Using temporal memory to stitch time-based graph computations
• Spiking Hyperdimensional Network: Neuromorphic Models Integrated with Memory-Inspired Framework
• Deep Spiking Neural Network with Spike Count based Learning Rule
• VSA: Reconfigurable Vectorwise Spiking Neural Network Accelerator
• LIAF-Net: Leaky Integrate and Analog Fire Network for Lightweight and Efficient Spatiotemporal Information Processing
• Fractional-order spike-timing-dependent gradient descent for multi-layer spiking neural networks
• Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network
• Fourier-based Action Recognition for Wildlife Behavior Quantification with Event Cameras
• Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inference
• SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features
• SpikePipe: Accelerated Training of Spiking Neural Networks via Inter-Layer Pipelining and Multiprocessor Scheduling
• A Spike Learning System for Event-driven Object Recognition
• Unifying Activation- and Timing-based Learning Rules for Spiking Neural Networks
• Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications
• A Study On the Effects of Pre-processing On Spatio-temporal Action Recognition Using Spiking Neural Networks Trained with STDP
• Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State
• A Low Latency Adaptive Coding Spiking Framework for Deep Reinforcement Learning
• MAP-SNN: Mapping Spike Activities with Multiplicity, Adaptability, and Plasticity into Bio-Plausible Spiking Neural Networks
• Training Energy-Efficient Deep Spiking Neural Networks with Time-to-First-Spike Coding
• Contemporary implementations of spiking bio-inspired neural networks
• Composing Recurrent Spiking Neural Networks using Locally-Recurrent Motifs and Risk-Mitigating Architectural Optimization
• Optimising Event-Driven Spiking Neural Network with Regularisation and Cutoff
• FastSpiker: Enabling Fast Training for Spiking Neural Networks on Event-based Data through Learning Rate Enhancements for Autonomous Embedded Systems
• Understanding the Functional Roles of Modelling Components in Spiking Neural Networks
• You Only Spike Once: Improving Energy-Efficient Neuromorphic Inference to ANN-Level Accuracy
• Tuning Convolutional Spiking Neural Network with Biologically-plausible Reward Propagation
• MK-SGN: A Spiking Graph Convolutional Network with Multimodal Fusion and Knowledge Distillation for Skeleton-based Action Recognition
• Bio-realistic and versatile artificial dendrites made of anti-ambipolar transistors
• STCSNN: High energy efficiency spike-train level spiking neural networks with spatio-temporal conversion
• TactileSGNet: A Spiking Graph Neural Network for Event-based Tactile Object Recognition
• Combining Spiking Neural Network and Artificial Neural Network for Enhanced Image Classification
• A Methodology to Study the Impact of Spiking Neural Network Parameters considering Event-Based Automotive Data
• Minibatch Processing in Spiking Neural Networks
• Robust and Efficient Adversarial Defense in SNNs via Image Purification and Joint Detection
• Attacking the Spike: On the Transferability and Security of Spiking Neural Networks to Adversarial Examples
• Towards free-response paradigm: a theory on decision-making in spiking neural networks
• Attention Mechanism with Energy-Friendly Operations
• A Fully Spiking Hybrid Neural Network for Energy-Efficient Object Detection
• Spiking Neural Networks with Single-Spike Temporal-Coded Neurons for Network Intrusion Detection
• Spiking Neural Networks -- Part I: Detecting Spatial Patterns
• PETNet -- Coincident Particle Event Detection using Spiking Neural Networks
• Gated Parametric Neuron for Spike-based Audio Recognition
• Event-Enhanced Multi-Modal Spiking Neural Network for Dynamic Obstacle Avoidance
• Improved robustness of reinforcement learning policies upon conversion to spiking neuronal network platforms applied to ATARI games
• Multivariate Time Series Classification Using Spiking Neural Networks
• Spiking Two-Stream Methods with Unsupervised STDP-based Learning for Action Recognition
• An Introduction to Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications
• Spiking CapsNet: A Spiking Neural Network With A Biologically Plausible Routing Rule Between Capsules
• Towards Efficient and Accurate Spiking Neural Networks via Adaptive Bit Allocation
• SpinAPS: A High-Performance Spintronic Accelerator for Probabilistic Spiking Neural Networks
• STISA: A 0.16 GOPS/W/PE Single-Shot Inference FPGA-based SNN Accelerator with Algorithm and Hardware Co-Design
• SFATTI: Spiking FPGA Accelerator for Temporal Task-driven Inference -- A Case Study on MNIST
• Replay4NCL: An Efficient Memory Replay-based Methodology for Neuromorphic Continual Learning in Embedded AI Systems
• BioLeaF: A Bio-plausible Learning Framework for Training of Spiking Neural Networks
• 2D versus 3D Convolutional Spiking Neural Networks Trained with Unsupervised STDP for Human Action Recognition
• Effects of Introducing Synaptic Scaling on Spiking Neural Network Learning
• Threshold Modulation for Online Test-Time Adaptation of Spiking Neural Networks
• 2022 Roadmap on Neuromorphic Computing and Engineering
• Temporal Surrogate Back-propagation for Spiking Neural Networks
• Training Dynamic Exponential Family Models with Causal and Lateral Dependencies for Generalized Neuromorphic Computing
• In-Hardware Learning of Multilayer Spiking Neural Networks on a Neuromorphic Processor