Transient chaotic dimensionality expansion by recurrent networks

Abstract

Neurons in the brain communicate with spikes, which are discrete events in time and value. Functional network models often employ rate units that are continuously coupled by analog signals. Is there a qualitative difference implied by these two forms of signaling? We develop a unified mean-field theory for large random networks to show that first- and second-order statistics in rate and binary networks are in fact identical if rate neurons receive the right amount of noise. Their response to presented stimuli, however, can be radically different. We quantify these differences by studying how nearby state trajectories evolve over time, asking to what extent the dynamics is chaotic. Chaos in the two models is found to be qualitatively different. In binary networks we find a network-size-dependent transition to chaos and a chaotic submanifold whose dimensionality expands stereotypically with time, while rate networks with matched statistics are nonchaotic. Dimensionality expansion in chaotic binary networks aids classification in reservoir computing and optimal performance is reached within about a single activation per neuron; a fast mechanism for computation that we demonstrate also in spiking networks. A generalization of this mechanism extends to rate networks in their respective chaotic regimes.

arXiv commentFinal, shortened post-print version. (see v3 for extended material on microscopic and macroscopic chaos)

References in corpus (30)

• Stimulus-Dependent Suppression of Chaos in Recurrent Neural Networks
• Decorrelation of neural-network activity by inhibitory feedback
• Transition to chaos in random neuronal networks
• Stimuli reduce the dimensionality of cortical activity
• Transition to chaos in random networks with cell-type-specific connectivity
• Universal properties of correlation transfer in integrate-and-fire neurons
• Topological and Dynamical Complexity of Random Neural Networks
• The correlation structure of local cortical networks intrinsically results from recurrent dynamics
• Master equations and the theory of stochastic path integrals
• Nonnormal amplification in random balanced neuronal networks
• Statistical field theory for neural networks
• Correlations and Synchrony in Threshold Neuron Models
• Optimal sequence memory in driven random networks
• Correlations between synapses in pairs of neurons slow down dynamics in randomly connected neural networks
• Path Integral Approach to Random Neural Networks
• Path integral methods for the dynamics of stochastic and disordered systems
• A unified view on weakly correlated recurrent networks
• Dynamics of interacting particle systems: stochastic process and field theory
• Edge of chaos and avalanches in neural networks with heavy-tailed synaptic weight distribution
• Chaos and reliability in balanced spiking networks with temporal drive
• Field theories and exact stochastic equations for interacting particle systems
• Dimensionality compression and expansion in Deep Neural Networks
• Statistical Mechanics of Recurrent Neural Networks II. Dynamics
• Revisiting chaos in stimulus-driven spiking networks: signal encoding and discrimination
• On the number of limit cycles in asymmetric neural networks
• Noise Suppression and Surplus Synchrony by Coincidence Detection
• Solving the 2-Dimensional Fokker-Planck Equation for Strongly Correlated Neurons
• Locking of correlated neural activity to ongoing oscillations
• Statistical mechanics of phase-space partitioning in large-scale spiking neuron circuits
• Transient amplification in balanced neural networks

Cited by in corpus (8)

• Dimension of activity in random neural networks
• Self-learning Machines based on Hamiltonian Echo Backpropagation
• Large Deviations Approach to Random Recurrent Neuronal Networks: Parameter Inference and Fluctuation-Induced Transitions
• Extended Anderson Criticality in Heavy-Tailed Neural Networks
• A Microscopic Theory of Intrinsic Timescales in Spiking Neural Networks
• Evolving to learn: discovering interpretable plasticity rules for spiking networks
• Discontinuous transition to chaos in a canonical random neural network
• Irreversibility in Non-reciprocal Chaotic Systems