Prominent characteristics of recurrent neuronal networks are robust against low synaptic weight resolution

Abstract

The representation of the natural-density, heterogeneous connectivity of neuronal network models at relevant spatial scales remains a challenge for Computational Neuroscience and Neuromorphic Computing. In particular, the memory demands imposed by the vast number of synapses in brain-scale network simulations constitutes a major obstacle. Limiting the number resolution of synaptic weights appears to be a natural strategy to reduce memory and compute load. In this study, we investigate the effects of a limited synaptic-weight resolution on the dynamics of recurrent spiking neuronal networks resembling local cortical circuits, and develop strategies for minimizing deviations from the dynamics of networks with high-resolution synaptic weights. We mimic the effect of a limited synaptic weight resolution by replacing normally distributed synaptic weights by weights drawn from a discrete distribution, and compare the resulting statistics characterizing firing rates, spike-train irregularity, and correlation coefficients with the reference solution. We show that a naive discretization of synaptic weights generally leads to a distortion of the spike-train statistics. Only if the weights are discretized such that the mean and the variance of the total synaptic input currents are preserved, the firing statistics remains unaffected for the types of networks considered in this study. For networks with sufficiently heterogeneous in-degrees, the firing statistics can be preserved even if all synaptic weights are replaced by the mean of the weight distribution. We conclude that even for simple networks with non-plastic neurons and synapses, a discretization of synaptic weights can lead to substantial deviations in the firing statistics, unless the discretization is performed with care and guided by a rigorous validation process.

arXiv comment39 pages, 8 figures, 5 tables

References in corpus (22)

• Deep Learning with Limited Numerical Precision
• Memory and information processing in neuromorphic systems
• A Survey on Methods and Theories of Quantized Neural Networks
• Full-density multi-scale account of structure and dynamics of macaque visual cortex
• Demonstrating Advantages of Neuromorphic Computation: A Pilot Study
• Distributions of covariances as a window into the operational regime of neuronal networks
• Two types of criticality in the brain
• Dynamical Entropy Production in Spiking Neuron Networks in the Balanced State
• The correlation structure of local cortical networks intrinsically results from recurrent dynamics
• Exploring the Connection Between Binary and Spiking Neural Networks
• Is a 4-bit synaptic weight resolution enough? - Constraints on enabling spike-timing dependent plasticity in neuromorphic hardware
• Breaking Synchrony by Heterogeneity in Complex Networks
• Scalability of asynchronous networks is limited by one-to-one mapping between effective connectivity and correlations
• Real-Time Cortical Simulation on Neuromorphic Hardware
• Connectivity Concepts in Neuronal Network Modeling
• Fast simulations of highly-connected spiking cortical models using GPUs
• Rounding Methods for Neural Networks with Low Resolution Synaptic Weights
• Modulated escape from a metastable state driven by colored noise
• Conditions for wave trains in spiking neural networks
• NESTML: a modeling language for spiking neurons
• Firing rate homeostasis counteracts changes in stability of recurrent neural networks caused by synapse loss in Alzheimer's disease
• The effect of heterogeneity on decorrelation mechanisms in spiking neural networks: a neuromorphic-hardware study

Cited by in corpus (5)

• Reconciliation of weak pairwise spike-train correlations and highly coherent local field potentials across space
• Runtime Construction of Large-Scale Spiking Neuronal Network Models on GPU Devices
• Evaluating the statistical similarity of neural network activity and connectivity via eigenvector angles
• Metadata practices for simulation workflows
• Phenomenological modeling of diverse and heterogeneous synaptic dynamics at natural density