Unsupervised Discovery of 3D Physical Objects from Video
arXiv:2007.12348
Abstract
We study the problem of unsupervised physical object discovery. While existing frameworks aim to decompose scenes into 2D segments based off each object's appearance, we explore how physics, especially object interactions, facilitates disentangling of 3D geometry and position of objects from video, in an unsupervised manner. Drawing inspiration from developmental psychology, our Physical Object Discovery Network (POD-Net) uses both multi-scale pixel cues and physical motion cues to accurately segment observable and partially occluded objects of varying sizes, and infer properties of those objects. Our model reliably segments objects on both synthetic and real scenes. The discovered object properties can also be used to reason about physical events.
arXiv commentICLR 2021; project webpage at http://yilundu.github.io/podnet
References in corpus (13)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Auto-Encoding Variational Bayes
- Semi-Supervised Learning with Deep Generative Models
- Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
- Multi-Object Representation Learning with Iterative Variational Inference
- MONet: Unsupervised Scene Decomposition and Representation
- Flexible Neural Representation for Physics Prediction
- Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics
- Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
- Learning Physical Intuition of Block Towers by Example
- Reasoning About Physical Interactions with Object-Oriented Prediction and Planning
- IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning
- R-SQAIR: Relational Sequential Attend, Infer, Repeat