ASP: Learning to Forget with Adaptive Synaptic Plasticity in Spiking Neural Networks

Abstract

A fundamental feature of learning in animals is the "ability to forget" that allows an organism to perceive, model and make decisions from disparate streams of information and adapt to changing environments. Against this backdrop, we present a novel unsupervised learning mechanism ASP (Adaptive Synaptic Plasticity) for improved recognition with Spiking Neural Networks (SNNs) for real time on-line learning in a dynamic environment. We incorporate an adaptive weight decay mechanism with the traditional Spike Timing Dependent Plasticity (STDP) learning to model adaptivity in SNNs. The leak rate of the synaptic weights is modulated based on the temporal correlation between the spiking patterns of the pre- and post-synaptic neurons. This mechanism helps in gradual forgetting of insignificant data while retaining significant, yet old, information. ASP, thus, maintains a balance between forgetting and immediate learning to construct a stable-plastic self-adaptive SNN for continuously changing inputs. We demonstrate that the proposed learning methodology addresses catastrophic forgetting while yielding significantly improved accuracy over the conventional STDP learning method for digit recognition applications. Additionally, we observe that the proposed learning model automatically encodes selective attention towards relevant features in the input data while eliminating the influence of background noise (or denoising) further improving the robustness of the ASP learning.

14 pages, 14 figures

References in corpus (1)

• Perovskite Quantum Organismoids

Cited by in corpus (13)

• Incremental Learning in Deep Convolutional Neural Networks Using Partial Network Sharing
• Recursive Least Squares with Variable-Direction Forgetting -- Compensating for the loss of persistency
• FSpiNN: An Optimization Framework for Memory- and Energy-Efficient Spiking Neural Networks
• Controlled Forgetting: Targeted Stimulation and Dopaminergic Plasticity Modulation for Unsupervised Lifelong Learning in Spiking Neural Networks
• High-parallelism Inception-like Spiking Neural Networks for Unsupervised Feature Learning
• lpSpikeCon: Enabling Low-Precision Spiking Neural Network Processing for Efficient Unsupervised Continual Learning on Autonomous Agents
• TopSpark: A Timestep Optimization Methodology for Energy-Efficient Spiking Neural Networks on Autonomous Mobile Agents
• Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications
• Towards Scalable, Efficient and Accurate Deep Spiking Neural Networks with Backward Residual Connections, Stochastic Softmax and Hybridization
• SpikeDyn: A Framework for Energy-Efficient Spiking Neural Networks with Continual and Unsupervised Learning Capabilities in Dynamic Environments
• Mantis: Enabling Energy-Efficient Autonomous Mobile Agents with Spiking Neural Networks
• Voltage-Driven Domain-Wall Motion based Neuro-Synaptic Devices for Dynamic On-line Learning
• Replay4NCL: An Efficient Memory Replay-based Methodology for Neuromorphic Continual Learning in Embedded AI Systems