Hierarchical Representation Learning for Markov Decision Processes
arXiv:2106.01655
Abstract
In this paper we present a novel method for learning hierarchical representations of Markov decision processes. Our method works by partitioning the state space into subsets, and defines subtasks for performing transitions between the partitions. We formulate the problem of partitioning the state space as an optimization problem that can be solved using gradient descent given a set of sampled trajectories, making our method suitable for high-dimensional problems with large state spaces. We empirically validate the method, by showing that it can successfully learn a useful hierarchical representation in a navigation domain. Once learned, the hierarchical representation can be used to solve different tasks in the given domain, thus generalizing knowledge across tasks.
References in corpus (11)
- Decoupled Weight Decay Regularization
- Action-Conditional Video Prediction using Deep Networks in Atari Games
- Go-Explore: a New Approach for Hard-Exploration Problems
- Automatic Goal Generation for Reinforcement Learning Agents
- Self-Imitation Learning
- A Laplacian Framework for Option Discovery in Reinforcement Learning
- Learning Parameterized Skills
- Multi-Level Discovery of Deep Options
- Near-Optimal Representation Learning for Hierarchical Reinforcement Learning
- Efficient Model-Based Deep Reinforcement Learning with Variational State Tabulation
- Option Discovery in Hierarchical Reinforcement Learning using Spatio-Temporal Clustering