Plan2Vec: Unsupervised Representation Learning by Latent Plans
arXiv:2005.03648
Abstract
In this paper we introduce plan2vec, an unsupervised representation learning approach that is inspired by reinforcement learning. Plan2vec constructs a weighted graph on an image dataset using near-neighbor distances, and then extrapolates this local metric to a global embedding by distilling path-integral over planned path. When applied to control, plan2vec offers a way to learn goal-conditioned value estimates that are accurate over long horizons that is both compute and sample efficient. We demonstrate the effectiveness of plan2vec on one simulated and two challenging real-world image datasets. Experimental results show that plan2vec successfully amortizes the planning cost, enabling reactive planning that is linear in memory and computation complexity rather than exhaustive over the entire state space.
code available at https://geyang.github.io/plan2vec
References in corpus (18)
- Adam: A Method for Stochastic Optimization
- Continuous control with deep reinforcement learning
- Playing Atari with Deep Reinforcement Learning
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Contrastive Multiview Coding
- Maximum a Posteriori Policy Optimisation
- Episodic Curiosity through Reachability
- Deep Successor Reinforcement Learning
- Learning Plannable Representations with Causal InfoGAN
- Skew-Fit: State-Covering Self-Supervised Reinforcement Learning
- Semi-parametric Topological Memory for Navigation
- Composable Planning with Attributes
- Self-supervised Learning of Image Embedding for Continuous Control
- TreeQN and ATreeC: Differentiable Tree-Structured Models for Deep Reinforcement Learning
- Learning to Manipulate Deformable Objects without Demonstrations
- Combining Q-Learning and Search with Amortized Value Estimates
- Robot Motion Planning in Learned Latent Spaces
- Gated Path Planning Networks