Learning to Compose Skills

Abstract

We present a differentiable framework capable of learning a wide variety of compositions of simple policies that we call skills. By recursively composing skills with themselves, we can create hierarchies that display complex behavior. Skill networks are trained to generate skill-state embeddings that are provided as inputs to a trainable composition function, which in turn outputs a policy for the overall task. Our experiments on an environment consisting of multiple collect and evade tasks show that this architecture is able to quickly build complex skills from simpler ones. Furthermore, the learned composition function displays some transfer to unseen combinations of skills, allowing for zero-shot generalizations.

Presented at NIPS 2017 Deep RL Symposium

References in corpus (3)

• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• Meta Learning Shared Hierarchies
• Environment-Independent Task Specifications via GLTL

Cited by in corpus (12)

• Multi-Task Reinforcement Learning with Soft Modularization
• Memorize or generalize? Searching for a compositional RNN in a haystack
• Learning Abstract Options
• Robust Conversational AI with Grounded Text Generation
• Composing Task-Agnostic Policies with Deep Reinforcement Learning
• Fast Adaptation via Policy-Dynamics Value Functions
• Solving Compositional Reinforcement Learning Problems via Task Reduction
• Efficient Deep Reinforcement Learning via Adaptive Policy Transfer
• Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulas
• Learning a natural-language to LTL executable semantic parser for grounded robotics
• Cooperative Learning of Disjoint Syntax and Semantics
• Biological Blueprints for Next Generation AI Systems