Self-supervised Visual Reinforcement Learning with Object-centric Representations
arXiv:2011.14381
Abstract
Autonomous agents need large repertoires of skills to act reasonably on new tasks that they have not seen before. However, acquiring these skills using only a stream of high-dimensional, unstructured, and unlabeled observations is a tricky challenge for any autonomous agent. Previous methods have used variational autoencoders to encode a scene into a low-dimensional vector that can be used as a goal for an agent to discover new skills. Nevertheless, in compositional/multi-object environments it is difficult to disentangle all the factors of variation into such a fixed-length representation of the whole scene. We propose to use object-centric representations as a modular and structured observation space, which is learned with a compositional generative world model. We show that the structure in the representations in combination with goal-conditioned attention policies helps the autonomous agent to discover and learn useful skills. These skills can be further combined to address compositional tasks like the manipulation of several different objects.
References in corpus (14)
- A Simple Framework for Contrastive Learning of Visual Representations
- Momentum Contrast for Unsupervised Visual Representation Learning
- Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
- QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
- Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots
- MONet: Unsupervised Scene Decomposition and Representation
- DARLA: Improving Zero-Shot Transfer in Reinforcement Learning
- Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
- COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration
- Planning with Goal-Conditioned Policies
- Disentangling the independently controllable factors of variation by interacting with the world
- Unsupervised Control Through Non-Parametric Discriminative Rewards
- Entity Abstraction in Visual Model-Based Reinforcement Learning
- A Perspective on Objects and Systematic Generalization in Model-Based RL