Formation of cell assemblies with iterative winners-take-all computation and excitation-inhibition balance

Abstract

This paper targets the problem of encoding information into binary cell assemblies. Spiking neural networks and k-winners-take-all models are two common approaches, but the first is hard to use for information processing and the second is too simple and lacks important features of the first. We present an intermediate model that shares the computational ease of kWTA and has more flexible and richer dynamics. It uses explicit inhibitory neurons to balance and shape excitation through an iterative procedure. This leads to a recurrent interaction between inhibitory and excitatory neurons that better adapts to the input distribution and performs such computations as habituation, decorrelation, and clustering. To show these, we investigate Hebbian-like learning rules and propose a new learning rule for binary weights with multiple stabilization mechanisms. Our source code is publicly available.

arXiv comment14 pages, 8 figures

References in corpus (6)

• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex
• How Can We Be So Dense? The Benefits of Using Highly Sparse Representations
• How do neurons operate on sparse distributed representations? A mathematical theory of sparsity, neurons and active dendrites
• Binary autoencoder with random binary weights
• Expressivity of expand-and-sparsify representations