Optimal Learning with Excitatory and Inhibitory synapses
arXiv:2005.12330 · doi:10.1371/journal.pcbi.1008536
Abstract
Characterizing the relation between weight structure and input/output statistics is fundamental for understanding the computational capabilities of neural circuits. In this work, I study the problem of storing associations between analog signals in the presence of correlations, using methods from statistical mechanics. I characterize the typical learning performance in terms of the power spectrum of random input and output processes. I show that optimal synaptic weight configurations reach a capacity of 0.5 for any fraction of excitatory to inhibitory weights and have a peculiar synaptic distribution with a finite fraction of silent synapses. I further provide a link between typical learning performance and principal components analysis in single cases. These results may shed light on the synaptic profile of brain circuits, such as cerebellar structures, that are thought to engage in processing time-dependent signals and performing on-line prediction.
arXiv comment16 pages, 5 figures
References in corpus (19)
- Supervised Learning in Spiking Neural Networks with FORCE Training
- The generalization error of random features regression: Precise asymptotics and double descent curve
- Replica Field Theory for Deterministic Models (II): A Non-Random Spin Glass with Glassy Behavior
- Balanced Excitation and Inhibition are Required for High-Capacity, Noise-Robust Neuronal Selectivity
- Classification and Geometry of General Perceptual Manifolds
- Modelling the influence of data structure on learning in neural networks: the hidden manifold model
- Mean-Field Equations for Spin Models with Orthogonal Interaction Matrices
- Analysis of CDMA systems that are characterized by eigenvalue spectrum
- Tractable approximations for probabilistic models: The adaptive TAP mean field approach
- Intrinsically-generated fluctuating activity in excitatory-inhibitory networks
- Generalisation error in learning with random features and the hidden manifold model
- Training dynamically balanced excitatory-inhibitory networks
- The role of the interaction matrix in mean-field spin glasses
- Perceptron capacity revisited: classification ability for correlated patterns
- Statistical learning theory of structured data
- Capacity-resolution trade-off in the optimal learning of multiple low-dimensional manifolds by attractor neural networks
- Counting the learnable functions of structured data
- Learning of correlated patterns by simple perceptrons
- Interactions between Intrinsic and Stimulus-Evoked Activity in Recurrent Neural Networks