Complexity without chaos: Plasticity within random recurrent networks generates robust timing and motor control

Abstract

It is widely accepted that the complex dynamics characteristic of recurrent neural circuits contributes in a fundamental manner to brain function. Progress has been slow in understanding and exploiting the computational power of recurrent dynamics for two main reasons: nonlinear recurrent networks often exhibit chaotic behavior and most known learning rules do not work in robust fashion in recurrent networks. Here we address both these problems by demonstrating how random recurrent networks (RRN) that initially exhibit chaotic dynamics can be tuned through a supervised learning rule to generate locally stable neural patterns of activity that are both complex and robust to noise. The outcome is a novel neural network regime that exhibits both transiently stable and chaotic trajectories. We further show that the recurrent learning rule dramatically increases the ability of RRNs to generate complex spatiotemporal motor patterns, and accounts for recent experimental data showing a decrease in neural variability in response to stimulus onset.

References in corpus (1)

• Clustering in Complex Directed Networks

Cited by in corpus (60)

• Physical reservoir computing -- An introductory perspective
• Recurrent Network Models Of Sequence Generation And Memory
• Artificial neural networks for neuroscientists: A primer
• Harnessing disordered quantum dynamics for machine learning
• Supervised Learning in Spiking Neural Networks with FORCE Training
• full-FORCE: A Target-Based Method for Training Recurrent Networks
• Controlling Recurrent Neural Networks by Conceptors
• Optimal sequence memory in driven random networks
• Intrinsically-generated fluctuating activity in excitatory-inhibitory networks
• Physics enhanced neural networks predict order and chaos
• Mesoscopic chaos mediated by Drude electron-hole plasma in silicon optomechanical oscillators
• Learning spatiotemporal signals using a recurrent spiking network that discretizes time
• A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning
• Designing spontaneous behavioral switching via chaotic itinerancy
• How single neuron properties shape chaotic dynamics and signal transmission in random neural networks
• Training dynamically balanced excitatory-inhibitory networks
• Using Firing-Rate Dynamics to Train Recurrent Networks of Spiking Model Neurons
• Revisiting chaos in stimulus-driven spiking networks: signal encoding and discrimination
• Chaos may enhance expressivity in cerebellar granular layer
• Impulse-induced localized control of chaos in starlike networks
• Randomly connected networks generate emergent selectivity and predict decoding properties of large populations of neurons
• Robust trajectory generation for robotic control on the neuromorphic research chip Loihi
• Interpretable Nonlinear Dynamic Modeling of Neural Trajectories
• Suppression of chaos in a partially driven recurrent neural network
• Instability to a heterogeneous oscillatory state in randomly connected recurrent networks with delayed interactions
• Emergence of event cascades in inhomogeneous networks
• Functional differentiations in evolutionary reservoir computing networks
• Target spiking patterns enable efficient and biologically plausible learning for complex temporal tasks
• Transient Chaos in BERT
• Oscillations enhance time-series prediction in reservoir computing with feedback
• From Biological Synapses to Intelligent Robots
• Composite FORCE learning of chaotic echo state networks for time-series prediction
• Structure of attractors in randomly connected networks
• Emergent Computations in Trained Artificial Neural Networks and Real Brains
• A Unifying Framework for Information Processing in Stochastically Driven Dynamical Systems
• Automatic exploration of structural regularities in networks
• Decoding Neuronal Networks: A Reservoir Computing Approach for Predicting Connectivity and Functionality
• Spontaneous and stimulus-induced coherent states of critically balanced neuronal networks
• Non-Markovian environment induced chaos in optomechanical system
• Statistical mechanics of phase-space partitioning in large-scale spiking neuron circuits
• One step back, two steps forward: interference and learning in recurrent neural networks
• Different eigenvalue distributions encode the same temporal tasks in recurrent neural networks
• Variational online learning of neural dynamics
• Autonomous learning and chaining of motor primitives using the Free Energy Principle
• Deep Continuous Networks
• Attractor-merging Crises and Intermittency in Reservoir Computing
• Feed-forward approximations to dynamic recurrent network architectures
• A model of reward-modulated motor learning with parallelcortical and basal ganglia pathways
• Gauge-covariant stochastic neural fields: Stability and finite-width effects
• Ideomotor feedback control in a recurrent neural network
• Multiple-timescale Neural Networks: Generation of Context-dependent Sequences and Inference through Autonomous Bifurcations
• Quadratic speedup of global search using a biased crossover of two good solutions
• A geometrical analysis of global stability in trained feedback networks
• Learning recurrent dynamics in spiking networks
• Chaos-guided Input Structuring for Improved Learning in Recurrent Neural Networks
• Encoding Sensory and Motor Patterns as Time-Invariant Trajectories in Recurrent Neural Networks
• Cyclic image generation using chaotic dynamics
• Impact of leaky dynamics on predictive path integration accuracy in recurrent neural networks
• A Predictive Coding Account for Chaotic Itinerancy
• Probing the relationship between linear dynamical systems and low-rank recurrent neural network models