The Logical Options Framework
arXiv:2102.12571
Abstract
Learning composable policies for environments with complex rules and tasks is a challenging problem. We introduce a hierarchical reinforcement learning framework called the Logical Options Framework (LOF) that learns policies that are satisfying, optimal, and composable. LOF efficiently learns policies that satisfy tasks by representing the task as an automaton and integrating it into learning and planning. We provide and prove conditions under which LOF will learn satisfying, optimal policies. And lastly, we show how LOF's learned policies can be composed to satisfy unseen tasks with only 10-50 retraining steps. We evaluate LOF on four tasks in discrete and continuous domains, including a 3D pick-and-place environment.
23 pages, 19 figures
References in corpus (8)
- Modular Multitask Reinforcement Learning with Policy Sketches
- PyRep: Bringing V-REP to Deep Robot Learning
- Modular Deep Reinforcement Learning with Temporal Logic Specifications
- A Composable Specification Language for Reinforcement Learning Tasks
- Planning With Uncertain Specifications (PUnS)
- A Survey of Knowledge-based Sequential Decision Making under Uncertainty
- What Should I Do Now? Marrying Reinforcement Learning and Symbolic Planning
- Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulas