Object-Centric Learning with Slot Attention

Abstract

Learning object-centric representations of complex scenes is a promising step towards enabling efficient abstract reasoning from low-level perceptual features. Yet, most deep learning approaches learn distributed representations that do not capture the compositional properties of natural scenes. In this paper, we present the Slot Attention module, an architectural component that interfaces with perceptual representations such as the output of a convolutional neural network and produces a set of task-dependent abstract representations which we call slots. These slots are exchangeable and can bind to any object in the input by specializing through a competitive procedure over multiple rounds of attention. We empirically demonstrate that Slot Attention can extract object-centric representations that enable generalization to unseen compositions when trained on unsupervised object discovery and supervised property prediction tasks.

arXiv commentNeurIPS 2020. Code available at https://github.com/google-research/google-research/tree/master/slot_attention

References in corpus (35)

• Natural Scales in Geographical Patterns
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• A Structured Self-attentive Sentence Embedding
• Dota 2 with Large Scale Deep Reinforcement Learning
• Recurrent Models of Visual Attention
• Interaction Networks for Learning about Objects, Relations and Physics
• Learning Representations and Generative Models for 3D Point Clouds
• MolGAN: An implicit generative model for small molecular graphs
• Learning to Simulate Complex Physics with Graph Networks
• Multi-Object Representation Learning with Iterative Variational Inference
• MONet: Unsupervised Scene Decomposition and Representation
• Flexible Neural Representation for Physics Prediction
• Relational Deep Reinforcement Learning
• GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations
• Unsupervised Learning of Object Keypoints for Perception and Control
• Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
• Learning Independent Causal Mechanisms
• SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition
• Recurrent Independent Mechanisms
• Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs
• COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration
• Deep Set Prediction Networks
• FSPool: Learning Set Representations with Featurewise Sort Pooling
• Investigating Object Compositionality in Generative Adversarial Networks
• Stochastic Prediction of Multi-Agent Interactions from Partial Observations
• Graph Density-Aware Losses for Novel Compositions in Scene Graph Generation
• SCALOR: Generative World Models with Scalable Object Representations
• Conditional Set Generation with Transformers
• Towards causal generative scene models via competition of experts
• R-SQAIR: Relational Sequential Attend, Infer, Repeat
• Capsules with Inverted Dot-Product Attention Routing
• Loss Functions for Multiset Prediction
• Competitive Training of Mixtures of Independent Deep Generative Models
• Better Set Representations For Relational Reasoning
• Learning to Manipulate Individual Objects in an Image

Cited by in corpus (27)

• ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases
• Multiscale Vision Transformers
• Vision Transformers Need Registers
• Discrete and continuous representations and processing in deep learning: Looking forward
• Deep Generative Models for Detector Signature Simulation: A Taxonomic Review
• Assaying Out-Of-Distribution Generalization in Transfer Learning
• Unsupervised part representation by Flow Capsules
• Generative Adversarial Transformers
• Emergent Symbols through Binding in External Memory
• Efficient Scopeformer: Towards Scalable and Rich Feature Extraction for Intracranial Hemorrhage Detection
• Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices
• Weakly Supervised Learning of Multi-Object 3D Scene Decompositions Using Deep Shape Priors
• Cross-Modal Conceptualization in Bottleneck Models
• Set-Conditional Set Generation for Particle Physics
• Learning Object-Centric Video Models by Contrasting Sets
• Visual Reasoning in Object-Centric Deep Neural Networks: A Comparative Cognition Approach
• Self-supervised Scene Text Segmentation with Object-centric Layered Representations Augmented by Text Regions
• Local Slot Attention for Vision-and-Language Navigation
• Reasoning-Modulated Representations
• Compositional Attention: Disentangling Search and Retrieval
• Focus on the Core: Efficient Attention via Pruned Token Compression for Document Classification
• Mini-Batch Consistent Slot Set Encoder for Scalable Set Encoding
• Set Prediction without Imposing Structure as Conditional Density Estimation
• Learning Multi-Object Symbols for Manipulation with Attentive Deep Effect Predictors
• Self-Labeling of Fully Mediating Representations by Graph Alignment
• Group Equivariant Subsampling
• Recurrent Attention Models with Object-centric Capsule Representation for Multi-object Recognition