Spiking neural networks trained with backpropagation for low power neuromorphic implementation of voice activity detection

Abstract

Recent advances in Voice Activity Detection (VAD) are driven by artificial and Recurrent Neural Networks (RNNs), however, using a VAD system in battery-operated devices requires further power efficiency. This can be achieved by neuromorphic hardware, which enables Spiking Neural Networks (SNNs) to perform inference at very low energy consumption. Spiking networks are characterized by their ability to process information efficiently, in a sparse cascade of binary events in time called spikes. However, a big performance gap separates artificial from spiking networks, mostly due to a lack of powerful SNN training algorithms. To overcome this problem we exploit an SNN model that can be recast into an RNN-like model and trained with known deep learning techniques. We describe an SNN training procedure that achieves low spiking activity and pruning algorithms to remove 85% of the network connections with no performance loss. The model achieves state-of-the-art performance with a fraction of power consumption comparing to other methods.

arXiv comment5 pages, 2 figures, 2 tables

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Deep Learning in Spiking Neural Networks
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• A Survey of Model Compression and Acceleration for Deep Neural Networks
• Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
• SuperSpike: Supervised learning in multi-layer spiking neural networks
• SLAYER: Spike Layer Error Reassignment in Time
• A 0.086-mm${}^2$ 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28nm CMOS
• Deep Rewiring: Training very sparse deep networks
• Surrogate Gradient Learning in Spiking Neural Networks
• Spiking neural networks trained with backpropagation for low power neuromorphic implementation of voice activity detection

Cited by in corpus (4)

• Pruning of Deep Spiking Neural Networks through Gradient Rewiring
• Spiking neural networks trained with backpropagation for low power neuromorphic implementation of voice activity detection
• In-Ear-Voice: Towards Milli-Watt Audio Enhancement With Bone-Conduction Microphones for In-Ear Sensing Platforms
• MLNET: An Adaptive Multiple Receptive-field Attention Neural Network for Voice Activity Detection