Follow the Object: Curriculum Learning for Manipulation Tasks with Imagined Goals
arXiv:2008.02066
Abstract
Learning robot manipulation through deep reinforcement learning in environments with sparse rewards is a challenging task. In this paper we address this problem by introducing a notion of imaginary object goals. For a given manipulation task, the object of interest is first trained to reach a desired target position on its own, without being manipulated, through physically realistic simulations. The object policy is then leveraged to build a predictive model of plausible object trajectories providing the robot with a curriculum of incrementally more difficult object goals to reach during training. The proposed algorithm, Follow the Object (FO), has been evaluated on 7 MuJoCo environments requiring increasing degree of exploration, and has achieved higher success rates compared to alternative algorithms. In particularly challenging learning scenarios, e.g. where the object's initial and target positions are far apart, our approach can still learn a policy whereas competing methods currently fail.
arXiv commentTo appear in Deep Reinforcement Learning Workshop, NeurIPS 2021
References in corpus (14)
- Continuous control with deep reinforcement learning
- Trust Region Policy Optimization
- Solving Rubik's Cube with a Robot Hand
- Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
- Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
- Large-Scale Study of Curiosity-Driven Learning
- Visual Reinforcement Learning with Imagined Goals
- Deep Q-learning from Demonstrations
- Exploration by Random Network Distillation
- Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research
- Episodic Curiosity through Reachability
- Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids
- Data-efficient Deep Reinforcement Learning for Dexterous Manipulation
- Reinforcement Learning for Robotic Manipulation using Simulated Locomotion Demonstrations