Unsupervised Discovery of Parts, Structure, and Dynamics

Abstract

Humans easily recognize object parts and their hierarchical structure by watching how they move; they can then predict how each part moves in the future. In this paper, we propose a novel formulation that simultaneously learns a hierarchical, disentangled object representation and a dynamics model for object parts from unlabeled videos. Our Parts, Structure, and Dynamics (PSD) model learns to, first, recognize the object parts via a layered image representation; second, predict hierarchy via a structural descriptor that composes low-level concepts into a hierarchical structure; and third, model the system dynamics by predicting the future. Experiments on multiple real and synthetic datasets demonstrate that our PSD model works well on all three tasks: segmenting object parts, building their hierarchical structure, and capturing their motion distributions.

arXiv commentICLR 2019. The first two authors contributed equally to this work

Cited by in corpus (12)

• Learning Physical Graph Representations from Visual Scenes
• Unsupervised part representation by Flow Capsules
• Unsupervised Learning of Object Structure and Dynamics from Videos
• Physics-as-Inverse-Graphics: Unsupervised Physical Parameter Estimation from Video
• CSNNs: Unsupervised, Backpropagation-free Convolutional Neural Networks for Representation Learning
• Benchmarking Unsupervised Object Representations for Video Sequences
• Unsupervised object-centric video generation and decomposition in 3D
• Unsupervised Part Discovery from Contrastive Reconstruction
• Causal World Models by Unsupervised Deconfounding of Physical Dynamics
• ModeRNN: Harnessing Spatiotemporal Mode Collapse in Unsupervised Predictive Learning
• Deep Variational Luenberger-type Observer for Stochastic Video Prediction
• Boxhead: A Dataset for Learning Hierarchical Representations