Linking connectivity, dynamics and computations in low-rank recurrent neural networks

Abstract

Large scale neural recordings have established that the transformation of sensory stimuli into motor outputs relies on low-dimensional dynamics at the population level, while individual neurons exhibit complex selectivity. Understanding how low-dimensional computations on mixed, distributed representations emerge from the structure of the recurrent connectivity and inputs to cortical networks is a major challenge. Here, we study a class of recurrent network models in which the connectivity is a sum of a random part and a minimal, low-dimensional structure. We show that, in such networks, the dynamics are low dimensional and can be directly inferred from connectivity using a geometrical approach. We exploit this understanding to determine minimal connectivity required to implement specific computations, and find that the dynamical range and computational capacity quickly increase with the dimensionality of the connectivity structure. This framework produces testable experimental predictions for the relationship between connectivity, low-dimensional dynamics and computational features of recorded neurons.

References in corpus (5)

• On the difficulty of training Recurrent Neural Networks
• Transition to chaos in random networks with cell-type-specific connectivity
• Learning universal computations with spikes
• Intrinsically-generated fluctuating activity in excitatory-inhibitory networks
• Local Dynamics in Trained Recurrent Neural Networks

Cited by in corpus (45)

• Artificial neural networks for neuroscientists: A primer
• Large-scale neural recordings call for new insights to link brain and behavior
• Dynamics of random recurrent networks with correlated low-rank structure
• Learning spatiotemporal signals using a recurrent spiking network that discretizes time
• The Geometry of Information Coding in Correlated Neural Populations
• How single neuron properties shape chaotic dynamics and signal transmission in random neural networks
• Training dynamically balanced excitatory-inhibitory networks
• Interpreting recurrent neural networks behaviour via excitable network attractors
• Dimension of activity in random neural networks
• Universality and individuality in neural dynamics across large populations of recurrent networks
• Contrasting the effects of adaptation and synaptic filtering on the timescales of dynamics in recurrent networks
• Description of spreading dynamics by microscopic network models and macroscopic branching processes can differ due to coalescence
• Interpretable statistical representations of neural population dynamics and geometry
• Macroscopic Fluctuations Emerge in Balanced Networks with Incomplete Recurrent Alignment
• Predictive coding in balanced neural networks with noise, chaos and delays
• Non-Hermitian Quasi-Localization and Ring Attractor Neural Networks
• Lattice physics approaches for neural networks
• Extended Anderson Criticality in Heavy-Tailed Neural Networks
• Dynamical Mean-Field Theory of Complex Systems on Sparse Directed Networks
• Slow manifolds in recurrent networks encode working memory efficiently and robustly
• Input correlations impede suppression of chaos and learning in balanced rate networks
• Information-driven transitions in projections of underdamped dynamics
• Volcano transition in populations of phase oscillators with random nonreciprocal interactions
• Effect of Synaptic Heterogeneity on Neuronal Coordination
• Ensemble perspective for understanding temporal credit assignment
• Decoding Neuronal Networks: A Reservoir Computing Approach for Predicting Connectivity and Functionality
• Discontinuous transition to chaos in a canonical random neural network
• Irreversibility in Non-reciprocal Chaotic Systems
• The impact of memory on learning sequence-to-sequence tasks
• Bayesian reconstruction of memories stored in neural networks from their connectivity
• Spiking mode-based neural networks
• Multi-band oscillations emerge from a simple spiking network
• Hierarchy of chaotic dynamics in random modular networks
• Signature of glassy dynamics in dynamic modes decompositions
• Nonlinear computations in spiking neural networks through multiplicative synapses
• Renormalization group for spectral collapse in random matrices with power-law variance profiles
• Category learning in deep neural networks: Information content and geometry of internal representations
• From the perceptron to the cerebellum
• Shaping dynamical neural computations using spatiotemporal constraints
• A brain basis of dynamical intelligence for AI and computational neuroscience
• Probing the relationship between linear dynamical systems and low-rank recurrent neural network models
• A geometrical analysis of global stability in trained feedback networks
• A ghost mechanism: An analytical model of abrupt learning in recurrent networks
• Low-Dimensional Manifolds Support Multiplexed Integrations in Recurrent Neural Networks
• Comparing Dynamical Models Through Diffeomorphic Vector Field Alignment