Hierarchical Reinforcement Learning with Hindsight
arXiv:1805.08180
Abstract
Reinforcement Learning (RL) algorithms can suffer from poor sample efficiency when rewards are delayed and sparse. We introduce a solution that enables agents to learn temporally extended actions at multiple levels of abstraction in a sample efficient and automated fashion. Our approach combines universal value functions and hindsight learning, allowing agents to learn policies belonging to different time scales in parallel. We show that our method significantly accelerates learning in a variety of discrete and continuous tasks.
arXiv commentDuplicate. See arXiv:1712.00948 "Learning Multi-Level Hierarchies with Hindsight" for latest version
References in corpus (4)
Cited by in corpus (17)
- A survey on intrinsic motivation in reinforcement learning
- CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning
- Search on the Replay Buffer: Bridging Planning and Reinforcement Learning
- Mapping State Space using Landmarks for Universal Goal Reaching
- Learning Compositional Neural Programs with Recursive Tree Search and Planning
- From Pixels to Legs: Hierarchical Learning of Quadruped Locomotion
- Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation
- Sub-policy Adaptation for Hierarchical Reinforcement Learning
- Hierarchical Policy Learning is Sensitive to Goal Space Design
- Hierarchical Reinforcement Learning for Quadruped Locomotion
- Hindsight Trust Region Policy Optimization
- Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks
- DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics
- Correcting Experience Replay for Multi-Agent Communication
- Learning Compositional Neural Programs for Continuous Control
- Efficient Robotic Object Search via HIEM: Hierarchical Policy Learning with Intrinsic-Extrinsic Modeling
- HAC Explore: Accelerating Exploration with Hierarchical Reinforcement Learning