Stochasticity from function -- why the Bayesian brain may need no noise

Abstract

An increasing body of evidence suggests that the trial-to-trial variability of spiking activity in the brain is not mere noise, but rather the reflection of a sampling-based encoding scheme for probabilistic computing. Since the precise statistical properties of neural activity are important in this context, many models assume an ad-hoc source of well-behaved, explicit noise, either on the input or on the output side of single neuron dynamics, most often assuming an independent Poisson process in either case. However, these assumptions are somewhat problematic: neighboring neurons tend to share receptive fields, rendering both their input and their output correlated; at the same time, neurons are known to behave largely deterministically, as a function of their membrane potential and conductance. We suggest that spiking neural networks may, in fact, have no need for noise to perform sampling-based Bayesian inference. We study analytically the effect of auto- and cross-correlations in functionally Bayesian spiking networks and demonstrate how their effect translates to synaptic interaction strengths, rendering them controllable through synaptic plasticity. This allows even small ensembles of interconnected deterministic spiking networks to simultaneously and co-dependently shape their output activity through learning, enabling them to perform complex Bayesian computation without any need for noise, which we demonstrate in silico, both in classical simulation and in neuromorphic emulation. These results close a gap between the abstract models and the biology of functionally Bayesian spiking networks, effectively reducing the architectural constraints imposed on physical neural substrates required to perform probabilistic computing, be they biological or artificial.

References in corpus (11)

• STDP-based spiking deep convolutional neural networks for object recognition
• SuperSpike: Supervised learning in multi-layer spiking neural networks
• EMNIST: an extension of MNIST to handwritten letters
• Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines
• Stochastic inference with spiking neurons in the high-conductance state
• Characterization and Compensation of Network-Level Anomalies in Mixed-Signal Neuromorphic Modeling Platforms
• Spatiotemporal dynamics and reliable computations in recurrent spiking neural networks
• Pattern representation and recognition with accelerated analog neuromorphic systems
• Deterministic networks for probabilistic computing
• Probabilistic inference in discrete spaces can be implemented into networks of LIF neurons
• Spike-based probabilistic inference with correlated noise

Cited by in corpus (13)

• Versatile emulation of spiking neural networks on an accelerated neuromorphic substrate
• Accelerated physical emulation of Bayesian inference in spiking neural networks
• Cortical oscillations implement a backbone for sampling-based computation in spiking neural networks
• Quantum computing with classical bits
• Spiking neuromorphic chip learns entangled quantum states
• Fast and energy-efficient neuromorphic deep learning with first-spike times
• SpikE: spike-based embeddings for multi-relational graph data
• The Discrete Langevin Machine: Bridging the Gap Between Thermodynamic and Neuromorphic Systems
• Accelerating SNN Training with Stochastic Parallelizable Spiking Neurons
• Evolving to learn: discovering interpretable plasticity rules for spiking networks
• Emulating quantum computation with artificial neural networks
• Coherent noise enables probabilistic sequence replay in spiking neuronal networks
• An Introduction to Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications