Emergence of functional and structural properties of the head direction system by optimization of recurrent neural networks

Abstract

Recent work suggests goal-driven training of neural networks can be used to model neural activity in the brain. While response properties of neurons in artificial neural networks bear similarities to those in the brain, the network architectures are often constrained to be different. Here we ask if a neural network can recover both neural representations and, if the architecture is unconstrained and optimized, the anatomical properties of neural circuits. We demonstrate this in a system where the connectivity and the functional organization have been characterized, namely, the head direction circuits of the rodent and fruit fly. We trained recurrent neural networks (RNNs) to estimate head direction through integration of angular velocity. We found that the two distinct classes of neurons observed in the head direction system, the Compass neurons and the Shifter neurons, emerged naturally in artificial neural networks as a result of training. Furthermore, connectivity analysis and in-silico neurophysiology revealed structural and mechanistic similarities between artificial networks and the head direction system. Overall, our results show that optimization of RNNs in a goal-driven task can recapitulate the structure and function of biological circuits, suggesting that artificial neural networks can be used to study the brain at the level of both neural activity and anatomical organization.

arXiv commentInternational Conference on Learning Representations (ICLR) 2020

References in corpus (5)

• Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Brain's Ventral Visual Pathway
• Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
• Emergence of grid-like representations by training recurrent neural networks to perform spatial localization
• Universality and individuality in neural dynamics across large populations of recurrent networks
• Randomly connected networks generate emergent selectivity and predict decoding properties of large populations of neurons

Cited by in corpus (4)

• Neural tuning and representational geometry
• Recurrent neural network models for working memory of continuous variables: activity manifolds, connectivity patterns, and dynamic codes
• On Path Integration of Grid Cells: Group Representation and Isotropic Scaling
• Hippocampal representations emerge when training recurrent neural networks on a memory dependent maze navigation task