Going in circles is the way forward: the role of recurrence in visual inference

Abstract

Biological visual systems exhibit abundant recurrent connectivity. State-of-the-art neural network models for visual recognition, by contrast, rely heavily or exclusively on feedforward computation. Any finite-time recurrent neural network (RNN) can be unrolled along time to yield an equivalent feedforward neural network (FNN). This important insight suggests that computational neuroscientists may not need to engage recurrent computation, and that computer-vision engineers may be limiting themselves to a special case of FNN if they build recurrent models. Here we argue, to the contrary, that FNNs are a special case of RNNs and that computational neuroscientists and engineers should engage recurrence to understand how brains and machines can (1) achieve greater and more flexible computational depth, (2) compress complex computations into limited hardware, (3) integrate priors and priorities into visual inference through expectation and attention, (4) exploit sequential dependencies in their data for better inference and prediction, and (5) leverage the power of iterative computation.

References in corpus (9)

• Deep Learning in Neural Networks: An Overview
• Sequence to Sequence Learning with Neural Networks
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
• Controversial stimuli: pitting neural networks against each other as models of human recognition
• Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs
• Highway and Residual Networks learn Unrolled Iterative Estimation
• Crowding Reveals Fundamental Differences in Local vs. Global Processing in Humans and Machines
• KerCNNs: biologically inspired lateral connections for classification of corrupted images

Cited by in corpus (13)

• Five Points to Check when Comparing Visual Perception in Humans and Machines
• Learning Physical Graph Representations from Visual Scenes
• Data-driven emergence of convolutional structure in neural networks
• Predictive Coding: a Theoretical and Experimental Review
• Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency
• Hierarchically Compositional Tasks and Deep Convolutional Networks
• Spiking representation learning for associative memories
• Layer Folding: Neural Network Depth Reduction using Activation Linearization
• How does the primate brain combine generative and discriminative computations in vision?
• Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation
• Hybrid Backpropagation Parallel Reservoir Networks
• Early Exiting Predictive Coding Neural Networks for Edge AI
• Recurrent Attention Models with Object-centric Capsule Representation for Multi-object Recognition