Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Brain's Ventral Visual Pathway

Abstract

Converging evidence suggests that the mammalian ventral visual pathway encodes increasingly complex stimulus features in downstream areas. Using deep convolutional neural networks, we can now quantitatively demonstrate that there is indeed an explicit gradient for feature complexity in the ventral pathway of the human brain. Our approach also allows stimulus features of increasing complexity to be mapped across the human brain, providing an automated approach to probing how representations are mapped across the cortical sheet. Finally, it is shown that deep convolutional neural networks allow decoding of representations in the human brain at a previously unattainable degree of accuracy, providing a more sensitive window into the human brain.

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Improving neural networks by preventing co-adaptation of feature detectors
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• Visualizing and Understanding Convolutional Networks
• Pixels to Voxels: Modeling Visual Representation in the Human Brain
• Data-driven HRF estimation for encoding and decoding models

Cited by in corpus (78)

• Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
• Neural network models and deep learning - a primer for biologists
• Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision
• BOLD5000: A public fMRI dataset of 5000 images
• Unsupervised deep learning identifies semantic disentanglement in single inferotemporal neurons
• Deep Networks Can Resemble Human Feed-forward Vision in Invariant Object Recognition
• Task-Driven Convolutional Recurrent Models of the Visual System
• Controversial stimuli: pitting neural networks against each other as models of human recognition
• Going in circles is the way forward: the role of recurrence in visual inference
• Modeling the dynamics of human brain activity with recurrent neural networks
• Decoding and mapping task states of the human brain via deep learning
• Towards an integration of deep learning and neuroscience
• From voxels to pixels and back: Self-supervision in natural-image reconstruction from fMRI
• Convolutional Sketch Inversion
• Deep Predictive Coding Network for Object Recognition
• Explanatory models in neuroscience: Part 1 -- taking mechanistic abstraction seriously
• Deep Neural Networks predict Hierarchical Spatio-temporal Cortical Dynamics of Human Visual Object Recognition
• Leveraging Deep Graph-Based Text Representation for Sentiment Polarity Applications
• Multiparametric Deep Learning Tissue Signatures for a Radiological Biomarker of Breast Cancer: Preliminary Results
• Pragmatic inference and visual abstraction enable contextual flexibility during visual communication
• Mind the gap: Challenges of deep learning approaches to Theory of Mind
• Visual pathways from the perspective of cost functions and multi-task deep neural networks
• Distinguishing mirror from glass: A 'big data' approach to material perception
• Mechanisms of dimensionality reduction and decorrelation in deep neural networks
• Emergence of functional and structural properties of the head direction system by optimization of recurrent neural networks
• Explanatory models in neuroscience: Part 2 -- constraint-based intelligibility
• Multimodal neural networks better explain multivoxel patterns in the hippocampus
• The Temporal Structure of Language Processing in the Human Brain Corresponds to The Layered Hierarchy of Deep Language Models
• Modeling biological face recognition with deep convolutional neural networks
• Brain-mediated Transfer Learning of Convolutional Neural Networks
• Semantic vector space models predict neural responses to complex visual stimuli
• An Explicit Local and Global Representation Disentanglement Framework with Applications in Deep Clustering and Unsupervised Object Detection
• Correspondence of Deep Neural Networks and the Brain for Visual Textures
• GAIT-prop: A biologically plausible learning rule derived from backpropagation of error
• Insightful analysis of historical sources at scales beyond human capabilities using unsupervised Machine Learning and XAI
• Guiding Visual Attention in Deep Convolutional Neural Networks Based on Human Eye Movements
• Basic Level Categorization Facilitates Visual Object Recognition
• If deep learning is the answer, then what is the question?
• Transfer entropy-based feedback improves performance in artificial neural networks
• Deep driven fMRI decoding of visual categories
• The Future of Data Analysis in the Neurosciences
• Category decoding of visual stimuli from human brain activity using a bidirectional recurrent neural network to simulate bidirectional information flows in human visual cortices
• Fast shared response model for fMRI data
• Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization
• Deep adversarial neural decoding
• Dissimilarity learning via Siamese network predicts brain imaging data
• Data augmentation and image understanding
• Deep Neural Models for color discrimination and color constancy
• Teaching CNNs to mimic Human Visual Cognitive Process & regularise Texture-Shape bias
• Brain2Music: Reconstructing Music from Human Brain Activity
• How does the primate brain combine generative and discriminative computations in vision?
• Effective and efficient ROI-wise visual encoding using an end-to-end CNN regression model and selective optimization
• Long-range and hierarchical language predictions in brains and algorithms
• Net2Brain: A Toolbox to compare artificial vision models with human brain responses
• The relationship between Biological and Artificial Intelligence
• Diverse feature visualizations reveal invariances in early layers of deep neural networks
• In silico discovery of representational relationships across visual cortex
• Neural encoding with visual attention
• Modeling Category-Selective Cortical Regions with Topographic Variational Autoencoders
• ImageNet-trained deep neural network exhibits illusion-like response to the Scintillating Grid
• Convolutional neural networks with extra-classical receptive fields
• Deep Recurrent Encoder: A scalable end-to-end network to model brain signals
• Do Deep Neural Networks Model Nonlinear Compositionality in the Neural Representation of Human-Object Interactions?
• Cognitive computational neuroscience
• Reconstructed spatial receptive field structures by reverse correlation technique explains the visual feature selectivity of units in deep convolutional neural networks
• Selective Particle Attention: Visual Feature-Based Attention in Deep Reinforcement Learning
• How Convolutional Neural Network Architecture Biases Learned Opponency and Colour Tuning
• Comparison Against Task Driven Artificial Neural Networks Reveals Functional Organization of Mouse Visual Cortex
• Crowding in humans is unlike that in convolutional neural networks
• Constraint-free Natural Image Reconstruction from fMRI Signals Based on Convolutional Neural Network
• Inferring brain-computational mechanisms with models of activity measurements
• An encoding framework with brain inner state for natural image identification
• More Than Meets the Eye: Self-Supervised Depth Reconstruction From Brain Activity
• BigGAN-based Bayesian reconstruction of natural images from human brain activity
• Multi-Objective Cognitive Model: a supervised approach for multi-subject fMRI analysis
• Relating CNNs with brain: Challenges and findings
• Learning to aggregate feature representations
• Understanding Information Processing in Human Brain by Interpreting Machine Learning Models