Disentangled Skill Embeddings for Reinforcement Learning
arXiv:1906.09223
Abstract
We propose a novel framework for multi-task reinforcement learning (MTRL). Using a variational inference formulation, we learn policies that generalize across both changing dynamics and goals. The resulting policies are parametrized by shared parameters that allow for transfer between different dynamics and goal conditions, and by task-specific latent-space embeddings that allow for specialization to particular tasks. We show how the latent-spaces enable generalization to unseen dynamics and goals conditions. Additionally, policies equipped with such embeddings serve as a space of skills (or options) for hierarchical reinforcement learning. Since we can change task dynamics and goals independently, we name our framework Disentangled Skill Embeddings (DSE).
References in corpus (12)
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Reinforcement Learning with Deep Energy-Based Policies
- Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
- Path integrals and symmetry breaking for optimal control theory
- Data-Efficient Hierarchical Reinforcement Learning
- Maximum a Posteriori Policy Optimisation
- Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research
- Meta-Reinforcement Learning of Structured Exploration Strategies
- Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
- Decoupling Dynamics and Reward for Transfer Learning
- Multi-task Deep Reinforcement Learning with PopArt
- Benchmark Environments for Multitask Learning in Continuous Domains
Cited by in corpus (6)
- Transfer Learning in Deep Reinforcement Learning: A Survey
- A Tutorial on Sparse Gaussian Processes and Variational Inference
- Fast Adaptation via Policy-Dynamics Value Functions
- A Foliated View of Transfer Learning
- Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learning
- Learning to Transfer: A Foliated Theory