Deep Neural Networks predict Hierarchical Spatio-temporal Cortical Dynamics of Human Visual Object Recognition

Abstract

The complex multi-stage architecture of cortical visual pathways provides the neural basis for efficient visual object recognition in humans. However, the stage-wise computations therein remain poorly understood. Here, we compared temporal (magnetoencephalography) and spatial (functional MRI) visual brain representations with representations in an artificial deep neural network (DNN) tuned to the statistics of real-world visual recognition. We showed that the DNN captured the stages of human visual processing in both time and space from early visual areas towards the dorsal and ventral streams. Further investigation of crucial DNN parameters revealed that while model architecture was important, training on real-world categorization was necessary to enforce spatio-temporal hierarchical relationships with the brain. Together our results provide an algorithmically informed view on the spatio-temporal dynamics of visual object recognition in the human visual brain.

15 pages, 6 figures

References in corpus (2)

• Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
• Pixels to Voxels: Modeling Visual Representation in the Human Brain

Cited by in corpus (8)

• Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision
• Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner
• Correspondence of Deep Neural Networks and the Brain for Visual Textures
• Transfer entropy-based feedback improves performance in artificial neural networks
• Hidden-Fold Networks: Random Recurrent Residuals Using Sparse Supermasks
• Deep Neural Models for color discrimination and color constancy
• Understanding Information Processing in Human Brain by Interpreting Machine Learning Models
• On the Selective and Invariant Representation of DCNN for High-Resolution Remote Sensing Image Recognition