Generalization and Robustness Implications in Object-Centric Learning
arXiv:2107.00637
Abstract
The idea behind object-centric representation learning is that natural scenes can better be modeled as compositions of objects and their relations as opposed to distributed representations. This inductive bias can be injected into neural networks to potentially improve systematic generalization and performance of downstream tasks in scenes with multiple objects. In this paper, we train state-of-the-art unsupervised models on five common multi-object datasets and evaluate segmentation metrics and downstream object property prediction. In addition, we study generalization and robustness by investigating the settings where either a single object is out of distribution -- e.g., having an unseen color, texture, or shape -- or global properties of the scene are altered -- e.g., by occlusions, cropping, or increasing the number of objects. From our experimental study, we find object-centric representations to be useful for downstream tasks and generally robust to most distribution shifts affecting objects. However, when the distribution shift affects the input in a less structured manner, robustness in terms of segmentation and downstream task performance may vary significantly across models and distribution shifts.
arXiv commentPublished at ICML 2022
References in corpus (32)
- Auto-Encoding Variational Bayes
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Dota 2 with Large Scale Deep Reinforcement Learning
- Recurrent Models of Visual Attention
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- Disentangling by Factorising
- Interaction Networks for Learning about Objects, Relations and Physics
- Multi-Object Representation Learning with Iterative Variational Inference
- MONet: Unsupervised Scene Decomposition and Representation
- GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations
- Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions
- Are Disentangled Representations Helpful for Abstract Visual Reasoning?
- Learning Deep Disentangled Embeddings with the F-Statistic Loss
- Structured Disentangled Representations
- Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
- SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition
- Recurrent Independent Mechanisms
- Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs
- On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset
- On the Binding Problem in Artificial Neural Networks
- On Disentangled Representations Learned From Correlated Data
- Investigating Object Compositionality in Generative Adversarial Networks
- SCALOR: Generative World Models with Scalable Object Representations
- Towards causal generative scene models via competition of experts
- Structured Object-Aware Physics Prediction for Video Modeling and Planning
- GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement
- Competitive Training of Mixtures of Independent Deep Generative Models
- Reconstruction Bottlenecks in Object-Centric Generative Models
- Slot Contrastive Networks: A Contrastive Approach for Representing Objects
- Learning Object-Centric Video Models by Contrasting Sets
- LAVAE: Disentangling Location and Appearance