Recurrent Attention Models with Object-centric Capsule Representation for Multi-object Recognition

Abstract

The visual system processes a scene using a sequence of selective glimpses, each driven by spatial and object-based attention. These glimpses reflect what is relevant to the ongoing task and are selected through recurrent processing and recognition of the objects in the scene. In contrast, most models treat attention selection and recognition as separate stages in a feedforward process. Here we show that using capsule networks to create an object-centric hidden representation in an encoder-decoder model with iterative glimpse attention yields effective integration of attention and recognition. We evaluate our model on three multi-object recognition tasks; highly overlapping digits, digits among distracting clutter and house numbers, and show that it learns to effectively move its glimpse window, recognize and reconstruct the objects, all with only the classification as supervision. Our work takes a step toward a general architecture for how to integrate recurrent object-centric representation into the planning of attentional glimpses.

References in corpus (23)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Multiple Object Recognition with Visual Attention
• Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
• Object-Centric Learning with Slot Attention
• MONet: Unsupervised Scene Decomposition and Representation
• Learning to detect chest radiographs containing lung nodules using visual attention networks
• Deep Networks with Internal Selective Attention through Feedback Connections
• Perceiver: General Perception with Iterative Attention
• Going in circles is the way forward: the role of recurrence in visual inference
• Dynamic Routing Between Capsules
• Recurrent Independent Mechanisms
• Spatial Transformer Networks
• On the Binding Problem in Artificial Neural Networks
• Capsule Networks with Max-Min Normalization
• Saccader: Improving Accuracy of Hard Attention Models for Vision
• Hierarchical Attentive Recurrent Tracking
• Deflecting Adversarial Attacks
• Disentangling neural mechanisms for perceptual grouping
• Hard-Attention for Scalable Image Classification
• Capsule Network is Not More Robust than Convolutional Network
• Discovering Dynamic Salient Regions for Spatio-Temporal Graph Neural Networks