COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration

Abstract

Data efficiency and robustness to task-irrelevant perturbations are long-standing challenges for deep reinforcement learning algorithms. Here we introduce a modular approach to addressing these challenges in a continuous control environment, without using hand-crafted or supervised information. Our Curious Object-Based seaRch Agent (COBRA) uses task-free intrinsically motivated exploration and unsupervised learning to build object-based models of its environment and action space. Subsequently, it can learn a variety of tasks through model-based search in very few steps and excel on structured hold-out tests of policy robustness.

References in corpus (14)

• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• A simple neural network module for relational reasoning
• Model-Based Reinforcement Learning for Atari
• FeUdal Networks for Hierarchical Reinforcement Learning
• Implicit Quantile Networks for Distributional Reinforcement Learning
• Maximum a Posteriori Policy Optimisation
• MONet: Unsupervised Scene Decomposition and Representation
• Distributional Reinforcement Learning with Quantile Regression
• Towards Deep Symbolic Reinforcement Learning
• SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning
• Diversity is All You Need: Learning Skills without a Reward Function
• Learning Plannable Representations with Causal InfoGAN
• Curiosity Driven Exploration of Learned Disentangled Goal Spaces
• Learning Powerful Policies by Using Consistent Dynamics Model

Cited by in corpus (30)

• Object-Centric Learning with Slot Attention
• Contrastive Learning of Structured World Models
• Rule-Based Reinforcement Learning for Efficient Robot Navigation with Space Reduction
• Unsupervised Model Selection for Variational Disentangled Representation Learning
• BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments
• Entity Abstraction in Visual Model-Based Reinforcement Learning
• Discrete and continuous representations and processing in deep learning: Looking forward
• Hamiltonian Generative Networks
• Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning
• R-SQAIR: Relational Sequential Attend, Infer, Repeat
• Structured Object-Aware Physics Prediction for Video Modeling and Planning
• Physion: Evaluating Physical Prediction from Vision in Humans and Machines
• Counterfactual Data Augmentation using Locally Factored Dynamics
• AlignNet: Unsupervised Entity Alignment
• ROLL: Visual Self-Supervised Reinforcement Learning with Object Reasoning
• Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction
• Reconstruction Bottlenecks in Object-Centric Generative Models
• Imagined Value Gradients: Model-Based Policy Optimization with Transferable Latent Dynamics Models
• Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
• Benchmarking Unsupervised Object Representations for Video Sequences
• Unsupervised Learning of Lagrangian Dynamics from Images for Prediction and Control
• Learning Visual Dynamics Models of Rigid Objects using Relational Inductive Biases
• Self-supervised Visual Reinforcement Learning with Object-centric Representations
• Beyond Tabula-Rasa: a Modular Reinforcement Learning Approach for Physically Embedded 3D Sokoban
• Causal World Models by Unsupervised Deconfounding of Physical Dynamics
• Joint Perception and Control as Inference with an Object-based Implementation
• Plug and Play, Model-Based Reinforcement Learning
• Temporally Abstract Partial Models
• Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environment
• Autonomous sPOMDP Environment Modeling With Partial Model Exploitation