SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition

Abstract

The ability to decompose complex multi-object scenes into meaningful abstractions like objects is fundamental to achieve higher-level cognition. Previous approaches for unsupervised object-oriented scene representation learning are either based on spatial-attention or scene-mixture approaches and limited in scalability which is a main obstacle towards modeling real-world scenes. In this paper, we propose a generative latent variable model, called SPACE, that provides a unified probabilistic modeling framework that combines the best of spatial-attention and scene-mixture approaches. SPACE can explicitly provide factorized object representations for foreground objects while also decomposing background segments of complex morphology. Previous models are good at either of these, but not both. SPACE also resolves the scalability problems of previous methods by incorporating parallel spatial-attention and thus is applicable to scenes with a large number of objects without performance degradations. We show through experiments on Atari and 3D-Rooms that SPACE achieves the above properties consistently in comparison to SPAIR, IODINE, and GENESIS. Results of our experiments can be found on our project website: https://sites.google.com/view/space-project-page

Accepted in ICLR 2020

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Playing Atari with Deep Reinforcement Learning
• Categorical Reparameterization with Gumbel-Softmax
• Multi-Object Representation Learning with Iterative Variational Inference
• MONet: Unsupervised Scene Decomposition and Representation
• GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations
• Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions
• Continuously Differentiable Exponential Linear Units
• Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
• Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs

Cited by in corpus (23)

• Object-Centric Learning with Slot Attention
• Unsupervised Discovery of Object Radiance Fields
• GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement
• Compositional Transformers for Scene Generation
• Generative Neurosymbolic Machines
• Weakly Supervised Learning of Multi-Object 3D Scene Decompositions Using Deep Shape Priors
• Attention over learned object embeddings enables complex visual reasoning
• Slot Contrastive Networks: A Contrastive Approach for Representing Objects
• Generalization and Robustness Implications in Object-Centric Learning
• Better Set Representations For Relational Reasoning
• Benchmarking Unsupervised Object Representations for Video Sequences
• Unsupervised object-centric video generation and decomposition in 3D
• SLASH: Embracing Probabilistic Circuits into Neural Answer Set Programming
• Unsupervised Part Discovery from Contrastive Reconstruction
• Boosting Object Representation Learning via Motion and Object Continuity
• Unsupervised Object Learning via Common Fate
• Language-Mediated, Object-Centric Representation Learning
• Polyline Generative Navigable Space Segmentation for Autonomous Visual Navigation
• MarioNette: Self-Supervised Sprite Learning
• Conditional Object-Centric Learning from Video
• Understanding in Artificial Intelligence
• Relevance-Guided Modeling of Object Dynamics for Reinforcement Learning
• Illiterate DALL-E Learns to Compose