Spiking representation learning for associative memories

Abstract

Networks of interconnected neurons communicating through spiking signals offer the bedrock of neural computations. Our brains spiking neural networks have the computational capacity to achieve complex pattern recognition and cognitive functions effortlessly. However, solving real-world problems with artificial spiking neural networks (SNNs) has proved to be difficult for a variety of reasons. Crucially, scaling SNNs to large networks and processing large-scale real-world datasets have been challenging, especially when compared to their non-spiking deep learning counterparts. The critical operation that is needed of SNNs is the ability to learn distributed representations from data and use these representations for perceptual, cognitive and memory operations. In this work, we introduce a novel SNN that performs unsupervised representation learning and associative memory operations leveraging Hebbian synaptic and activity-dependent structural plasticity coupled with neuron-units modelled as Poisson spike generators with sparse firing (~1 Hz mean and ~100 Hz maximum firing rate). Crucially, the architecture of our model derives from the neocortical columnar organization and combines feedforward projections for learning hidden representations and recurrent projections for forming associative memories. We evaluated the model on properties relevant for attractor-based associative memories such as pattern completion, perceptual rivalry, distortion resistance, and prototype extraction.

References in corpus (8)

• Deep Learning in Spiking Neural Networks
• Recurrence is required to capture the representational dynamics of the human visual system
• Generalisation in humans and deep neural networks
• Recurrent computations for visual pattern completion
• Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks
• Going in circles is the way forward: the role of recurrence in visual inference
• Classification and Generation of real-world data with an Associative Memory Model
• Benchmarking Hebbian learning rules for associative memory

Cited by in corpus (1)

• Benchmarking local Hebbian learning rules for memory storage and prototype extraction