Adaptive Skills, Adaptive Partitions (ASAP)

Abstract

We introduce the Adaptive Skills, Adaptive Partitions (ASAP) framework that (1) learns skills (i.e., temporally extended actions or options) as well as (2) where to apply them. We believe that both (1) and (2) are necessary for a truly general skill learning framework, which is a key building block needed to scale up to lifelong learning agents. The ASAP framework can also solve related new tasks simply by adapting where it applies its existing learned skills. We prove that ASAP converges to a local optimum under natural conditions. Finally, our experimental results, which include a RoboCup domain, demonstrate the ability of ASAP to learn where to reuse skills as well as solve multiple tasks with considerably less experience than solving each task from scratch.

References in corpus (3)

• Hierarchical Solution of Markov Decision Processes using Macro-actions
• Compositional Planning Using Optimal Option Models
• Safe Policy Search for Lifelong Reinforcement Learning with Sublinear Regret

Cited by in corpus (11)

• A Deep Hierarchical Approach to Lifelong Learning in Minecraft
• Eigenoption Discovery through the Deep Successor Representation
• An empirical investigation of the challenges of real-world reinforcement learning
• Learning Abstract Options
• Safe Option-Critic: Learning Safety in the Option-Critic Architecture
• Discovering Options for Exploration by Minimizing Cover Time
• Interpretable Reinforcement Learning Inspired by Piaget's Theory of Cognitive Development
• Discovery of Options via Meta-Learned Subgoals
• A Matrix Splitting Perspective on Planning with Options
• Options of Interest: Temporal Abstraction with Interest Functions
• When should agents explore?