Optimal reinforcement learning near the edge of synchronization transition

Abstract

Recent experimental and theoretical studies have indicated that the putative criticality of cortical dynamics may corresponds to a synchronization phase transition. The critical dynamics near such a critical point needs further investigation specifically when compared to the critical behavior near the standard absorbing state phase transition. Since the phenomena of learning and self-organized criticality (SOC) at the edge of synchronization transition can emerge jointly in spiking neural networks due to the presence of spike-timing dependent plasticity (STDP), it is tempting to ask: What is the relationship between synchronization and learning in neural networks? Further, does learning benefit from SOC at the edge of synchronization transition? In this paper, we intend to address these important issues. Accordingly, we construct a biologically inspired model of an autonomous cognitive system which learns to perform stimulus-response tasks. We train this system using a reinforcement learning rule implemented through dopamine-modulated STDP. We find that the system exhibits a continuous transition from synchronous to asynchronous neural oscillations upon increasing the average axonal time delay. We characterize the learning performance of the system and observe that it is optimized near the synchronization transition. We also study neuronal avalanches in the system and provide evidence that optimized learning is achieved in a slightly supercritical state.

arXiv comment9 pages, 5 figures. To appear in Physical Review E

References in corpus (15)

• Optimal Dynamical Range of Excitable Networks at Criticality
• Dynamical synapses causing self-organized criticality in neural networks
• Colloquium: Criticality and dynamical scaling in living systems
• Brain organization into resting state networks emerges at criticality on a model of the human connectome
• Self-Organized Criticality model for Brain Plasticity
• Landau-Ginzburg theory of cortex dynamics: Scale-free avalanches emerge at the edge of synchronization
• Griffiths phases on complex networks
• Adaptive self-organization in a realistic neural network model
• Control of criticality and computation in spiking neuromorphic networks with plasticity
• Complex and Adaptive Dynamical Systems: A Primer
• Growing Critical: Self-Organized Criticality in a Developing Neural System
• Beta-rhythm oscillations and synchronization transition in network models of Izhikevich neurons: effect of topology and synaptic type
• Spike-timing-dependent plasticity with axonal delay tunes networks of Izhikevich neurons to the edge of synchronization transition with scale-free avalanches
• Explosive, continuous and frustrated synchronization transition in spiking Hodgkin-Huxley neuronal networks: the role of topology and synaptic interaction
• Spatial features of synaptic adaptation affecting learning performance