Recurrent computations for visual pattern completion

Abstract

Making inferences from partial information constitutes a critical aspect of cognition. During visual perception, pattern completion enables recognition of poorly visible or occluded objects. We combined psychophysics, physiology and computational models to test the hypothesis that pattern completion is implemented by recurrent computations and present three pieces of evidence that are consistent with this hypothesis. First, subjects robustly recognized objects even when rendered <15% visible, but recognition was largely impaired when processing was interrupted by backward masking. Second, invasive physiological responses along the human ventral cortex exhibited visually selective responses to partially visible objects that were delayed compared to whole objects, suggesting the need for additional computations. These physiological delays were correlated with the effects of backward masking. Third, state-of-the-art feed-forward computational architectures were not robust to partial visibility. However, recognition performance was recovered when the model was augmented with attractor-based recurrent connectivity. These results provide a strong argument of plausibility for the role of recurrent computations in making visual inferences from partial information.

References in corpus (2)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding

Cited by in corpus (30)

• Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
• Recurrence is required to capture the representational dynamics of the human visual system
• Mass Ejection in Failed Supernovae: Variation with Stellar Progenitor
• Going in circles is the way forward: the role of recurrence in visual inference
• Learning Physical Graph Representations from Visual Scenes
• Crowding Reveals Fundamental Differences in Local vs. Global Processing in Humans and Machines
• Robustness of Object Recognition under Extreme Occlusion in Humans and Computational Models
• Gradient-free activation maximization for identifying effective stimuli
• Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency
• Look Twice: A Generalist Computational Model Predicts Return Fixations across Tasks and Species
• Disentangling neural mechanisms for perceptual grouping
• Guiding Visual Attention in Deep Convolutional Neural Networks Based on Human Eye Movements
• Hierarchical Predictive Coding Models in a Deep-Learning Framework
• Random Sampling Neural Network for Quantum Many-Body Problems
• KerCNNs: biologically inspired lateral connections for classification of corrupted images
• Stable and expressive recurrent vision models
• Convolutional Bipartite Attractor Networks
• Robust neural circuit reconstruction from serial electron microscopy with convolutional recurrent networks
• Energy--Information Trade-off Induces Continuous and Discontinuous Phase Transitions in Lateral Predictive Coding
• Hierarchically Compositional Tasks and Deep Convolutional Networks
• Spiking representation learning for associative memories
• Lateral predictive coding revisited: Internal model, symmetry breaking, and response time
• Deep Continuous Networks
• Putting visual object recognition in context
• What takes the brain so long: Object recognition at the level of minimal images develops for up to seconds of presentation time
• Seeing in the dark with recurrent convolutional neural networks
• Iterative VAE as a predictive brain model for out-of-distribution generalization
• Recurrent Connectivity Aids Recognition of Partly Occluded Objects
• Understanding Character Recognition using Visual Explanations Derived from the Human Visual System and Deep Networks
• Lift-the-flap: what, where and when for context reasoning