ROLL: Visual Self-Supervised Reinforcement Learning with Object Reasoning

Abstract

Current image-based reinforcement learning (RL) algorithms typically operate on the whole image without performing object-level reasoning. This leads to inefficient goal sampling and ineffective reward functions. In this paper, we improve upon previous visual self-supervised RL by incorporating object-level reasoning and occlusion reasoning. Specifically, we use unknown object segmentation to ignore distractors in the scene for better reward computation and goal generation; we further enable occlusion reasoning by employing a novel auxiliary loss and training scheme. We demonstrate that our proposed algorithm, ROLL (Reinforcement learning with Object Level Learning), learns dramatically faster and achieves better final performance compared with previous methods in several simulated visual control tasks. Project video and code are available at https://sites.google.com/andrew.cmu.edu/roll.

CoRL 2020. The first two authors contributed equally. Project video and code are available at https://sites.google.com/andrew.cmu.edu/roll

References in corpus (9)

• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Hindsight Experience Replay
• Relational Deep Reinforcement Learning
• Towards Deep Symbolic Reinforcement Learning
• Self-Supervised Visual Planning with Temporal Skip Connections
• COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration
• Reinforcement Learning of Active Vision for Manipulating Objects under Occlusions
• Unseen Object Instance Segmentation for Robotic Environments
• Reinforcement Learning without Ground-Truth State

Cited by in corpus (2)

• Weakly Supervised Disentangled Representation for Goal-conditioned Reinforcement Learning
• Self-supervised Reinforcement Learning with Independently Controllable Subgoals