Going in circles is the way forward: the role of recurrence in visual inference

Abstract

Biological visual systems exhibit abundant recurrent connectivity. State-of-the-art neural network models for visual recognition, by contrast, rely heavily or exclusively on feedforward computation. Any finite-time recurrent neural network (RNN) can be unrolled along time to yield an equivalent feedforward neural network (FNN). This important insight suggests that computational neuroscientists may not need to engage recurrent computation, and that computer-vision engineers may be limiting themselves to a special case of FNN if they build recurrent models. Here we argue, to the contrary, that FNNs are a special case of RNNs and that computational neuroscientists and engineers should engage recurrence to understand how brains and machines can (1) achieve greater and more flexible computational depth, (2) compress complex computations into limited hardware, (3) integrate priors and priorities into visual inference through expectation and attention, (4) exploit sequential dependencies in their data for better inference and prediction, and (5) leverage the power of iterative computation.

References in corpus (30)

• Deep Learning in Neural Networks: An Overview
• Neural Machine Translation by Jointly Learning to Align and Translate
• Sequence to Sequence Learning with Neural Networks
• Intriguing properties of neural networks
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Conditional Random Fields as Recurrent Neural Networks
• Unsupervised Learning of Video Representations using LSTMs
• Reconciling modern machine learning practice and the bias-variance trade-off
• Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Brain's Ventral Visual Pathway
• Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
• Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition
• Recurrence is required to capture the representational dynamics of the human visual system
• Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
• Adversarial Examples Are Not Bugs, They Are Features
• Video (language) modeling: a baseline for generative models of natural videos
• Recurrent computations for visual pattern completion
• Towards the first adversarially robust neural network model on MNIST
• Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex
• Deep Networks Can Resemble Human Feed-forward Vision in Invariant Object Recognition
• Task-Driven Convolutional Recurrent Models of the Visual System
• Controversial stimuli: pitting neural networks against each other as models of human recognition
• Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs
• Highway and Residual Networks learn Unrolled Iterative Estimation
• Residual Connections Encourage Iterative Inference
• Deep Predictive Coding Network for Object Recognition
• Crowding Reveals Fundamental Differences in Local vs. Global Processing in Humans and Machines
• Reviving and Improving Recurrent Back-Propagation
• Are Generative Classifiers More Robust to Adversarial Attacks?
• KerCNNs: biologically inspired lateral connections for classification of corrupted images
• Learning long-range spatial dependencies with horizontal gated-recurrent units

Cited by in corpus (13)

• Five Points to Check when Comparing Visual Perception in Humans and Machines
• Learning Physical Graph Representations from Visual Scenes
• Data-driven emergence of convolutional structure in neural networks
• Predictive Coding: a Theoretical and Experimental Review
• Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency
• Hierarchically Compositional Tasks and Deep Convolutional Networks
• Spiking representation learning for associative memories
• Layer Folding: Neural Network Depth Reduction using Activation Linearization
• How does the primate brain combine generative and discriminative computations in vision?
• Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation
• Hybrid Backpropagation Parallel Reservoir Networks
• Early Exiting Predictive Coding Neural Networks for Edge AI
• Recurrent Attention Models with Object-centric Capsule Representation for Multi-object Recognition