Spiking Deep Networks with LIF Neurons

Abstract

We train spiking deep networks using leaky integrate-and-fire (LIF) neurons, and achieve state-of-the-art results for spiking networks on the CIFAR-10 and MNIST datasets. This demonstrates that biologically-plausible spiking LIF neurons can be integrated into deep networks can perform as well as other spiking models (e.g. integrate-and-fire). We achieved this result by softening the LIF response function, such that its derivative remains bounded, and by training the network with noise to provide robustness against the variability introduced by spikes. Our method is general and could be applied to other neuron types, including those used on modern neuromorphic hardware. Our work brings more biological realism into modern image classification models, with the hope that these models can inform how the brain performs this difficult task. It also provides new methods for training deep networks to run on neuromorphic hardware, with the aim of fast, power-efficient image classification for robotics applications.

Cited by in corpus (50)

• Deep Learning in Spiking Neural Networks
• Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks
• STDP-based spiking deep convolutional neural networks for object recognition
• Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures
• The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks
• Neuromorphic Deep Learning Machines
• Deep Residual Learning in Spiking Neural Networks
• A Biologically Plausible Supervised Learning Method for Spiking Neural Networks Using the Symmetric STDP Rule
• Surrogate Gradient Learning in Spiking Neural Networks
• Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines
• Probabilistic Deep Spiking Neural Systems Enabled by Magnetic Tunnel Junction
• Training Spiking Deep Networks for Neuromorphic Hardware
• Theory and Tools for the Conversion of Analog to Spiking Convolutional Neural Networks
• Whetstone: A Method for Training Deep Artificial Neural Networks for Binary Communication
• BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning
• Spiking Deep Residual Network
• Deep Spiking Networks
• Gridbot: An autonomous robot controlled by a Spiking Neural Network mimicking the brain's navigational system
• Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks
• Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation
• RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network
• Simple and complex spiking neurons: perspectives and analysis in a simple STDP scenario
• An On-chip Trainable and Clock-less Spiking Neural Network with 1R Memristive Synapses
• Efficient Spiking Neural Networks with Logarithmic Temporal Coding
• A Tandem Learning Rule for Effective Training and Rapid Inference of Deep Spiking Neural Networks
• Optimal ANN-SNN Conversion for Fast and Accurate Inference in Deep Spiking Neural Networks
• SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural Networks
• Sneaky Spikes: Uncovering Stealthy Backdoor Attacks in Spiking Neural Networks with Neuromorphic Data
• Spiking Hyperdimensional Network: Neuromorphic Models Integrated with Memory-Inspired Framework
• Differentiable architecture search with multi-dimensional attention for spiking neural networks
• Unifying Activation- and Timing-based Learning Rules for Spiking Neural Networks
• DCT-SNN: Using DCT to Distribute Spatial Information over Time for Learning Low-Latency Spiking Neural Networks
• Towards Scalable, Efficient and Accurate Deep Spiking Neural Networks with Backward Residual Connections, Stochastic Softmax and Hybridization
• Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State
• A Spike in Performance: Training Hybrid-Spiking Neural Networks with Quantized Activation Functions
• Accelerating SNN Training with Stochastic Parallelizable Spiking Neurons
• Spike-based computation using classical recurrent neural networks
• A Neuromorphic Paradigm for Online Unsupervised Clustering
• Sparse Spiking Gradient Descent
• Deep counter networks for asynchronous event-based processing
• An All-Memristor Deep Spiking Neural Computing System: A Step Towards Realizing the Low Power,Stochastic Brain
• One Timestep is All You Need: Training Spiking Neural Networks with Ultra Low Latency
• RRAM based neuromorphic algorithms
• Spiking Neural Networks with Single-Spike Temporal-Coded Neurons for Network Intrusion Detection
• A temporally and spatially local spike-based backpropagation algorithm to enable training in hardware
• A Temporal Neural Network Architecture for Online Learning
• Bifurcation Spiking Neural Network
• SPENSER: Towards a NeuroEvolutionary Approach for Convolutional Spiking Neural Networks
• Neuromorphic Acceleration for Approximate Bayesian Inference on Neural Networks via Permanent Dropout
• On exploiting the synaptic interaction properties to obtain frequency-specific neurons