CLOUD: Contrastive Learning of Unsupervised Dynamics
arXiv:2010.12488
Abstract
Developing agents that can perform complex control tasks from high dimensional observations such as pixels is challenging due to difficulties in learning dynamics efficiently. In this work, we propose to learn forward and inverse dynamics in a fully unsupervised manner via contrastive estimation. Specifically, we train a forward dynamics model and an inverse dynamics model in the feature space of states and actions with data collected from random exploration. Unlike most existing deterministic models, our energy-based model takes into account the stochastic nature of agent-environment interactions. We demonstrate the efficacy of our approach across a variety of tasks including goal-directed planning and imitation from observations. Project videos and code are at https://jianrenw.github.io/cloud/.
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- A Simple Framework for Contrastive Learning of Visual Representations
- Improved Baselines with Momentum Contrastive Learning
- Contrastive Multiview Coding
- DeepMind Control Suite
- VIME: Variational Information Maximizing Exploration
- Model-Based Reinforcement Learning for Atari
- Learning Latent Dynamics for Planning from Pixels
- Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
- Learning Visual Predictive Models of Physics for Playing Billiards
- Learning Plannable Representations with Causal InfoGAN
- Learning Predictive Representations for Deformable Objects Using Contrastive Estimation
- Imitating Latent Policies from Observation