Network Plasticity as Bayesian Inference

Abstract

General results from statistical learning theory suggest to understand not only brain computations, but also brain plasticity as probabilistic inference. But a model for that has been missing. We propose that inherently stochastic features of synaptic plasticity and spine motility enable cortical networks of neurons to carry out probabilistic inference by sampling from a posterior distribution of network configurations. This model provides a viable alternative to existing models that propose convergence of parameters to maximum likelihood values. It explains how priors on weight distributions and connection probabilities can be merged optimally with learned experience, how cortical networks can generalize learned information so well to novel experiences, and how they can compensate continuously for unforeseen disturbances of the network. The resulting new theory of network plasticity explains from a functional perspective a number of experimental data on stochastic aspects of synaptic plasticity that previously appeared to be quite puzzling.

33 pages, 5 figures, the supplement is available on the author's web page http://www.igi.tugraz.at/kappel

References in corpus (1)

• Improving neural networks by preventing co-adaptation of feature detectors

Cited by in corpus (24)

• Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures
• Adaptive Extreme Edge Computing for Wearable Devices
• Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines
• An Introduction to Probabilistic Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications
• Demonstration of Decentralized, Physics-Driven Learning
• Synaptic metaplasticity in binarized neural networks
• Towards an integration of deep learning and neuroscience
• Pruning of Deep Spiking Neural Networks through Gradient Rewiring
• Efficient Reward-Based Structural Plasticity on a SpiNNaker 2 Prototype
• Multi-contact synapses for stable networks: a spike-timing dependent model of dendritic spine plasticity and turnover
• Desynchronous Learning in a Physics-Driven Learning Network
• The Physical Effects of Learning
• Deterministic networks for probabilistic computing
• Embodied Synaptic Plasticity with Online Reinforcement learning
• VOWEL: A Local Online Learning Rule for Recurrent Networks of Probabilistic Spiking Winner-Take-All Circuits
• Evolving to learn: discovering interpretable plasticity rules for spiking networks
• CaMKII activation supports reward-based neural network optimization through Hamiltonian sampling
• Control of the Correlation of Spontaneous Neuron Activity in Biological and Noise-activated CMOS Artificial Neural Microcircuits
• A dynamic connectome supports the emergence of stable computational function of neural circuits through reward-based learning
• Chance, long tails, and inference: a non-Gaussian, Bayesian theory of vocal learning in songbirds
• An Introduction to Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications
• Phenomenological modeling of diverse and heterogeneous synaptic dynamics at natural density
• Locally Learned Synaptic Dropout for Complete Bayesian Inference
• Training Dynamic Exponential Family Models with Causal and Lateral Dependencies for Generalized Neuromorphic Computing