Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future

Abstract

Convolutional neural networks (CNNs) were inspired by early findings in the study of biological vision. They have since become successful tools in computer vision and state-of-the-art models of both neural activity and behavior on visual tasks. This review highlights what, in the context of CNNs, it means to be a good model in computational neuroscience and the various ways models can provide insight. Specifically, it covers the origins of CNNs and the methods by which we validate them as models of biological vision. It then goes on to elaborate on what we can learn about biological vision by understanding and experimenting on CNNs and discusses emerging opportunities for the use of CNNS in vision research beyond basic object recognition.

arXiv commentReview Article to be published in Journal of Cognitive Neuroscience, 18 pages, 5 figures plus 8 pages of references

References in corpus (34)

• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Brain's Ventral Visual Pathway
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Understanding Deep Convolutional Networks
• Predicting Parameters in Deep Learning
• Towards Explainable Artificial Intelligence
• Neural network models and deep learning - a primer for biologists
• Similarity of Neural Network Representations Revisited
• Generalisation in humans and deep neural networks
• Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
• Recurrent computations for visual pattern completion
• On the importance of single directions for generalization
• BOLD5000: A public fMRI dataset of 5000 images
• Deep Learning Models of the Retinal Response to Natural Scenes
• Comparing deep neural networks against humans: object recognition when the signal gets weaker
• Deep Networks Can Resemble Human Feed-forward Vision in Invariant Object Recognition
• Classification and Geometry of General Perceptual Manifolds
• Controversial stimuli: pitting neural networks against each other as models of human recognition
• Reinforcement and Imitation Learning for Diverse Visuomotor Skills
• Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures
• Revisiting the Importance of Individual Units in CNNs via Ablation
• Local Aggregation for Unsupervised Learning of Visual Embeddings
• Hierarchical visuomotor control of humanoids
• Effects of Degradations on Deep Neural Network Architectures
• Attentional Neural Network: Feature Selection Using Cognitive Feedback
• Visual pathways from the perspective of cost functions and multi-task deep neural networks
• What does it mean to understand a neural network?
• A Connectome Based Hexagonal Lattice Convolutional Network Model of the Drosophila Visual System
• One-Shot High-Fidelity Imitation: Training Large-Scale Deep Nets with RL
• Invariant recognition drives neural representations of action sequences
• How can deep learning advance computational modeling of sensory information processing?
• Input Switched Affine Networks: An RNN Architecture Designed for Interpretability
• KerCNNs: biologically inspired lateral connections for classification of corrupted images
• The functional role of cue-driven feature-based feedback in object recognition

Cited by in corpus (27)

• Unsupervised deep learning identifies semantic disentanglement in single inferotemporal neurons
• Do Vision Transformers See Like Convolutional Neural Networks?
• 2022 Review of Data-Driven Plasma Science
• Detecting Visual Design Principles in Art and Architecture through Deep Convolutional Neural Networks
• A Spiking Neural Network for Image Segmentation
• CDLNet: Noise-Adaptive Convolutional Dictionary Learning Network for Blind Denoising and Demosaicing
• Bilevel methods for image reconstruction
• PreCNet: Next-Frame Video Prediction Based on Predictive Coding
• Improving deep learning with prior knowledge and cognitive models: A survey on enhancing explainability, adversarial robustness and zero-shot learning
• Testing the limits of natural language models for predicting human language judgments
• Spiking Hyperdimensional Network: Neuromorphic Models Integrated with Memory-Inspired Framework
• Learning representations in Bayesian Confidence Propagation neural networks
• Towards Biologically Plausible Convolutional Networks
• If deep learning is the answer, then what is the question?
• High-performance Evolutionary Algorithms for Online Neuron Control
• Inconsistent illusory motion in predictive coding deep neural networks
• Mutual influence between language and perception in multi-agent communication games
• Attention module improves both performance and interpretability of 4D fMRI decoding neural network
• SincVAE: A new semi-supervised approach to improve anomaly detection on EEG data using SincNet and variational autoencoder
• Playing With Neuroscience: Past, Present and Future of Neuroimaging and Games
• Deep Spiking Neural Networks with Resonate-and-Fire Neurons
• Supervised dimensionality reduction by a Linear Discriminant Analysis on pre-trained CNN features
• Immuno-mimetic Deep Neural Networks (Immuno-Net)
• Robust 3D U-Net Segmentation of Macular Holes
• How saccadic vision might help with theinterpretability of deep networks
• Understanding Attention: In Minds and Machines
• Understanding top-down attention using task-oriented ablation design