Contrastive Variational Reinforcement Learning for Complex Observations
arXiv:2008.02430
Abstract
Deep reinforcement learning (DRL) has achieved significant success in various robot tasks: manipulation, navigation, etc. However, complex visual observations in natural environments remains a major challenge. This paper presents Contrastive Variational Reinforcement Learning (CVRL), a model-based method that tackles complex visual observations in DRL. CVRL learns a contrastive variational model by maximizing the mutual information between latent states and observations discriminatively, through contrastive learning. It avoids modeling the complex observation space unnecessarily, as the commonly used generative observation model often does, and is significantly more robust. CVRL achieves comparable performance with state-of-the-art model-based DRL methods on standard Mujoco tasks. It significantly outperforms them on Natural Mujoco tasks and a robot box-pushing task with complex observations, e.g., dynamic shadows. The CVRL code is available publicly at https://github.com/Yusufma03/CVRL.
arXiv commentCoRL 2020 camera ready
References in corpus (12)
- Auto-Encoding Variational Bayes
- Playing Atari with Deep Reinforcement Learning
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- A Recurrent Latent Variable Model for Sequential Data
- DeepMind Control Suite
- Recurrent World Models Facilitate Policy Evolution
- Deep Variational Information Bottleneck
- Distributed Distributional Deterministic Policy Gradients
- Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids
- Dream to Control: Learning Behaviors by Latent Imagination
- Value Prediction Network
- Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations