Learning Contact-Rich Manipulation Skills with Guided Policy Search

Abstract

Autonomous learning of object manipulation skills can enable robots to acquire rich behavioral repertoires that scale to the variety of objects found in the real world. However, current motion skill learning methods typically restrict the behavior to a compact, low-dimensional representation, limiting its expressiveness and generality. In this paper, we extend a recently developed policy search method \cite{la-lnnpg-14} and use it to learn a range of dynamic manipulation behaviors with highly general policy representations, without using known models or example demonstrations. Our approach learns a set of trajectories for the desired motion skill by using iteratively refitted time-varying linear models, and then unifies these trajectories into a single control policy that can generalize to new situations. To enable this method to run on a real robot, we introduce several improvements that reduce the sample count and automate parameter selection. We show that our method can acquire fast, fluent behaviors after only minutes of interaction time, and can learn robust controllers for complex tasks, including putting together a toy airplane, stacking tight-fitting lego blocks, placing wooden rings onto tight-fitting pegs, inserting a shoe tree into a shoe, and screwing bottle caps onto bottles.

References in corpus (1)

• Exploring Deep and Recurrent Architectures for Optimal Control

Cited by in corpus (17)

• End-to-End Training of Deep Visuomotor Policies
• Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Control
• Learning to Optimize
• Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement Learning
• Preparing for the Unknown: Learning a Universal Policy with Online System Identification
• On the Geometry of Adversarial Examples
• Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstration
• HIRL: Hierarchical Inverse Reinforcement Learning for Long-Horizon Tasks with Delayed Rewards
• DoorGym: A Scalable Door Opening Environment And Baseline Agent
• Guided Policy Search as Approximate Mirror Descent
• Path Integral Guided Policy Search
• One-Shot Learning of Manipulation Skills with Online Dynamics Adaptation and Neural Network Priors
• DyNODE: Neural Ordinary Differential Equations for Dynamics Modeling in Continuous Control
• User Experience of the CoSTAR System for Instruction of Collaborative Robots
• Learning Deep Neural Network Policies with Continuous Memory States
• Variational Generative Stochastic Networks with Collaborative Shaping
• Multiform Adaptive Robot Skill Learning from Humans