Unsupervised Learning of Object Keypoints for Perception and Control

Abstract

The study of object representations in computer vision has primarily focused on developing representations that are useful for image classification, object detection, or semantic segmentation as downstream tasks. In this work we aim to learn object representations that are useful for control and reinforcement learning (RL). To this end, we introduce Transporter, a neural network architecture for discovering concise geometric object representations in terms of keypoints or image-space coordinates. Our method learns from raw video frames in a fully unsupervised manner, by transporting learnt image features between video frames using a keypoint bottleneck. The discovered keypoints track objects and object parts across long time-horizons more accurately than recent similar methods. Furthermore, consistent long-term tracking enables two notable results in control domains -- (1) using the keypoint co-ordinates and corresponding image features as inputs enables highly sample-efficient reinforcement learning; (2) learning to explore by controlling keypoint locations drastically reduces the search space, enabling deep exploration (leading to states unreachable through random action exploration) without any extrinsic rewards.

arXiv commentIn NeurIPS 2019. Code https://github.com/deepmind/deepmind-research/tree/master/transporter

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• Deep Convolutional Inverse Graphics Network
• DeepMind Control Suite
• Distributed Prioritized Experience Replay
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• Parameter Space Noise for Exploration
• Multi-Object Representation Learning with Iterative Variational Inference
• Go-Explore: a New Approach for Hard-Exploration Problems
• MONet: Unsupervised Scene Decomposition and Representation
• Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
• Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning
• Self-supervised learning of a facial attribute embedding from video
• Unsupervised Learning of Object Landmarks through Conditional Image Generation
• Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance
• Unsupervised Video Object Segmentation for Deep Reinforcement Learning
• Understanding Visual Concepts with Continuation Learning

Cited by in corpus (29)

• Object-Centric Learning with Slot Attention
• Causal Discovery in Physical Systems from Videos
• Deep Reinforcement Learning at the Edge of the Statistical Precipice
• Transporter Networks: Rearranging the Visual World for Robotic Manipulation
• Entity Abstraction in Visual Model-Based Reinforcement Learning
• 3D Neural Scene Representations for Visuomotor Control
• Induction and Exploitation of Subgoal Automata for Reinforcement Learning
• Augmented Skeleton Based Contrastive Action Learning with Momentum LSTM for Unsupervised Action Recognition
• Unsupervised Learning of Object Structure and Dynamics from Videos
• Deep Reinforcement Learning Based on Local GNN for Goal-conditioned Deformable Object Rearranging
• The Treachery of Images: Bayesian Scene Keypoints for Deep Policy Learning in Robotic Manipulation
• Improving Generative Imagination in Object-Centric World Models
• AlignNet: Unsupervised Entity Alignment
• Model-Based Inverse Reinforcement Learning from Visual Demonstrations
• Weakly Supervised Disentangled Representation for Goal-conditioned Reinforcement Learning
• Graph-Transporter: A Graph-based Learning Method for Goal-Conditioned Deformable Object Rearranging Task
• Learning Temporally Causal Latent Processes from General Temporal Data
• Unsupervised Learning of Landmarks based on Inter-Intra Subject Consistencies
• Unsupervised Learning of Lagrangian Dynamics from Images for Prediction and Control
• Vision-based system identification and 3D keypoint discovery using dynamics constraints
• Causal World Models by Unsupervised Deconfounding of Physical Dynamics
• GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation
• No Representation without Transformation
• Unsupervised Salient Patch Selection for Data-Efficient Reinforcement Learning
• Joint Perception and Control as Inference with an Object-based Implementation
• Learning of feature points without additional supervision improves reinforcement learning from images
• Pushing it out of the Way: Interactive Visual Navigation
• Learning Transferable Concepts in Deep Reinforcement Learning
• Relevance-Guided Modeling of Object Dynamics for Reinforcement Learning