Acceleration of Actor-Critic Deep Reinforcement Learning for Visual Grasping in Clutter by State Representation Learning Based on Disentanglement of a Raw Input Image
arXiv:2002.11903
Abstract
For a robotic grasping task in which diverse unseen target objects exist in a cluttered environment, some deep learning-based methods have achieved state-of-the-art results using visual input directly. In contrast, actor-critic deep reinforcement learning (RL) methods typically perform very poorly when grasping diverse objects, especially when learning from raw images and sparse rewards. To make these RL techniques feasible for vision-based grasping tasks, we employ state representation learning (SRL), where we encode essential information first for subsequent use in RL. However, typical representation learning procedures are unsuitable for extracting pertinent information for learning the grasping skill, because the visual inputs for representation learning, where a robot attempts to grasp a target object in clutter, are extremely complex. We found that preprocessing based on the disentanglement of a raw input image is the key to effectively capturing a compact representation. This enables deep RL to learn robotic grasping skills from highly varied and diverse visual inputs. We demonstrate the effectiveness of this approach with varying levels of disentanglement in a realistic simulated environment.
References in corpus (10)
- Adam: A Method for Stochastic Optimization
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- YOLOv3: An Incremental Improvement
- QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
- DeepMind Control Suite
- Reinforcement Learning with Unsupervised Auxiliary Tasks
- SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning
- Data-efficient Deep Reinforcement Learning for Dexterous Manipulation
- Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model
- Incremental Slow Feature Analysis: Adaptive and Episodic Learning from High-Dimensional Input Streams