Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Search

Abstract

In principle, reinforcement learning and policy search methods can enable robots to learn highly complex and general skills that may allow them to function amid the complexity and diversity of the real world. However, training a policy that generalizes well across a wide range of real-world conditions requires far greater quantity and diversity of experience than is practical to collect with a single robot. Fortunately, it is possible for multiple robots to share their experience with one another, and thereby, learn a policy collectively. In this work, we explore distributed and asynchronous policy learning as a means to achieve generalization and improved training times on challenging, real-world manipulation tasks. We propose a distributed and asynchronous version of Guided Policy Search and use it to demonstrate collective policy learning on a vision-based door opening task using four robots. We show that it achieves better generalization, utilization, and training times than the single robot alternative.

Submitted to the IEEE International Conference on Robotics and Automation 2017

References in corpus (7)

• Continuous control with deep reinforcement learning
• Trust Region Policy Optimization
• End-to-End Training of Deep Visuomotor Policies
• Revisiting Distributed Synchronous SGD
• Going Further with Point Pair Features
• Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates
• Guided Policy Search as Approximate Mirror Descent

Cited by in corpus (28)

• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
• Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control
• Data-efficient Deep Reinforcement Learning for Dexterous Manipulation
• Reinforcement and Imitation Learning for Diverse Visuomotor Skills
• Out of Distribution Generalization in Machine Learning
• An empirical investigation of the challenges of real-world reinforcement learning
• DiNNO: Distributed Neural Network Optimization for Multi-Robot Collaborative Learning
• COG: Connecting New Skills to Past Experience with Offline Reinforcement Learning
• Few-Shot Goal Inference for Visuomotor Learning and Planning
• Rigid-Soft Interactive Learning for Robust Grasping
• Bayesian policy selection using active inference
• From Pixels to Legs: Hierarchical Learning of Quadruped Locomotion
• Deep Predictive Policy Training using Reinforcement Learning
• In-Hand Object Stabilization by Independent Finger Control
• Capability-based Frameworks for Industrial Robot Skills: a Survey
• PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training
• Cooperative Control of Mobile Robots with Stackelberg Learning
• Automating Reinforcement Learning with Example-based Resets
• OffWorld Gym: open-access physical robotics environment for real-world reinforcement learning benchmark and research
• Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder
• A Workflow for Offline Model-Free Robotic Reinforcement Learning
• Mass-spring-damper Networks for Distributed Optimization in Non-Euclidean Spaces
• Unlocking Pixels for Reinforcement Learning via Implicit Attention
• Peer-Assisted Robotic Learning: A Data-Driven Collaborative Learning Approach for Cloud Robotic Systems
• RLC Circuits based Distributed Mirror Descent Method
• Neural-iLQR: A Learning-Aided Shooting Method for Trajectory Optimization
• Reinforcement Learning in Topology-based Representation for Human Body Movement with Whole Arm Manipulation
• Vision-based Robotic Arm Imitation by Human Gesture