Recurrence is required to capture the representational dynamics of the human visual system

Abstract

The human visual system is an intricate network of brain regions that enables us to recognize the world around us. Despite its abundant lateral and feedback connections, object processing is commonly viewed and studied as a feedforward process. Here, we measure and model the rapid representational dynamics across multiple stages of the human ventral stream using time-resolved brain imaging and deep learning. We observe substantial representational transformations during the first 300 ms of processing within and across ventral-stream regions. Categorical divisions emerge in sequence, cascading forward and in reverse across regions, and Granger causality analysis suggests bidirectional information flow between regions. Finally, recurrent deep neural network models clearly outperform parameter-matched feedforward models in terms of their ability to capture the multi-region cortical dynamics. Targeted virtual cooling experiments on the recurrent deep network models further substantiate the importance of their lateral and top-down connections. These results establish that recurrent models are required to understand information processing in the human ventral stream.

arXiv commenthttps://www.pnas.org/content/early/2019/10/04/1905544116.short?rss=1

References in corpus (7)

• TensorFlow: A system for large-scale machine learning
• Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
• Recurrent computations for visual pattern completion
• Task-Driven Convolutional Recurrent Models of the Visual System
• Recurrent Connections Aid Occluded Object Recognition by Discounting Occluders
• Deep Predictive Coding Network with Local Recurrent Processing for Object Recognition
• Robust neural circuit reconstruction from serial electron microscopy with convolutional recurrent networks

Cited by in corpus (22)

• Artificial neural networks for neuroscientists: A primer
• Going in circles is the way forward: the role of recurrence in visual inference
• An algebraic approach to the Kuramoto model
• Geometry unites synchrony, chimeras, and waves in nonlinear oscillator networks
• Predictive Coding: a Theoretical and Experimental Review
• Neural Networks with Recurrent Generative Feedback
• Neural Language Taskonomy: Which NLP Tasks are the most Predictive of fMRI Brain Activity?
• Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency
• Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics
• The Topology and Geometry of Neural Representations
• Guiding Visual Attention in Deep Convolutional Neural Networks Based on Human Eye Movements
• If deep learning is the answer, then what is the question?
• Neural tuning and representational geometry
• Mutual influence between language and perception in multi-agent communication games
• Spiking representation learning for associative memories
• How does the primate brain combine generative and discriminative computations in vision?
• GLiT: Neural Architecture Search for Global and Local Image Transformer
• Net2Brain: A Toolbox to compare artificial vision models with human brain responses
• Iterative VAE as a predictive brain model for out-of-distribution generalization
• Investigating Fine- and Coarse-grained Structural Correspondences Between Deep Neural Networks and Human Object Image Similarity Judgments Using Unsupervised Alignment
• Recurrent networks improve neural response prediction and provide insights into underlying cortical circuits
• Implicit recurrent networks: A novel approach to stationary input processing with recurrent neural networks in deep learning