Multi-task Reinforcement Learning with a Planning Quasi-Metric
arXiv:2002.03240
Abstract
We introduce a new reinforcement learning approach combining a planning quasi-metric (PQM) that estimates the number of steps required to go from any state to another, with task-specific "aimers" that compute a target state to reach a given goal. This decomposition allows the sharing across tasks of a task-agnostic model of the quasi-metric that captures the environment's dynamics and can be learned in a dense and unsupervised manner. We achieve multiple-fold training speed-up compared to recently published methods on the standard bit-flip problem and in the MuJoCo robotic arm simulator.
arXiv commentDeep RL Workshop, NeurIPS 2020
References in corpus (9)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Continuous control with deep reinforcement learning
- Hindsight Experience Replay
- Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research
- Planning with Goal-Conditioned Policies
- Search on the Replay Buffer: Bridging Planning and Reinforcement Learning
- Self-supervised Learning of Image Embedding for Continuous Control
- Multi-task Deep Reinforcement Learning with PopArt
- Floyd-Warshall Reinforcement Learning: Learning from Past Experiences to Reach New Goals