Unsupervised Object Learning via Common Fate
arXiv:2110.06562
Abstract
Learning generative object models from unlabelled videos is a long standing problem and required for causal scene modeling. We decompose this problem into three easier subtasks, and provide candidate solutions for each of them. Inspired by the Common Fate Principle of Gestalt Psychology, we first extract (noisy) masks of moving objects via unsupervised motion segmentation. Second, generative models are trained on the masks of the background and the moving objects, respectively. Third, background and foreground models are combined in a conditional "dead leaves" scene model to sample novel scene configurations where occlusions and depth layering arise naturally. To evaluate the individual stages, we introduce the Fishbowl dataset positioned between complex real-world scenes and common object-centric benchmarks of simplistic objects. We show that our approach allows learning generative models that generalize beyond the occlusions present in the input videos, and represent scenes in a modular fashion that allows sampling plausible scenes outside the training distribution by permitting, for instance, object numbers or densities not observed in the training set.
arXiv commentPublished at CLeaR 2023
References in corpus (19)
- Auto-Encoding Variational Bayes
- Improved Regularization of Convolutional Neural Networks with Cutout
- A guide to convolution arithmetic for deep learning
- Recurrent Models of Visual Attention
- Which Training Methods for GANs do actually Converge?
- Multi-Object Representation Learning with Iterative Variational Inference
- LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation
- MONet: Unsupervised Scene Decomposition and Representation
- GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations
- BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images
- Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
- Learning Independent Causal Mechanisms
- SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition
- The Informed Sampler: A Discriminative Approach to Bayesian Inference in Generative Computer Vision Models
- RELATE: Physically Plausible Multi-Object Scene Synthesis Using Structured Latent Spaces
- Towards causal generative scene models via competition of experts
- Learning Object-Centric Representations of Multi-Object Scenes from Multiple Views
- GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement
- LAVAE: Disentangling Location and Appearance