DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneck

Abstract

Deep reinforcement learning (DRL) agents are often sensitive to visual changes that were unseen in their training environments. To address this problem, we leverage the sequential nature of RL to learn robust representations that encode only task-relevant information from observations based on the unsupervised multi-view setting. Specifically, we introduce a novel contrastive version of the Multi-View Information Bottleneck (MIB) objective for temporal data. We train RL agents from pixels with this auxiliary objective to learn robust representations that can compress away task-irrelevant information and are predictive of task-relevant dynamics. This approach enables us to train high-performance policies that are robust to visual distractions and can generalize well to unseen environments. We demonstrate that our approach can achieve SOTA performance on a diverse set of visual control tasks in the DeepMind Control Suite when the background is replaced with natural videos. In addition, we show that our approach outperforms well-established baselines for generalization to unseen environments on the Procgen benchmark. Our code is open-sourced and available at https://github. com/BU-DEPEND-Lab/DRIBO.

arXiv commentAccepted at ICML 2022, 29 pages

References in corpus (17)

• End to End Learning for Self-Driving Cars
• The Kinetics Human Action Video Dataset
• The information bottleneck method
• Learning deep representations by mutual information estimation and maximization
• Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• DeepMind Control Suite
• Learning Latent Dynamics for Planning from Pixels
• A Study on Overfitting in Deep Reinforcement Learning
• On Variational Bounds of Mutual Information
• Generalization and Regularization in DQN
• Policy Transfer with Strategy Optimization
• Automatic Data Augmentation for Generalization in Deep Reinforcement Learning
• Natural Environment Benchmarks for Reinforcement Learning
• EMI: Exploration with Mutual Information
• Deep Reinforcement and InfoMax Learning
• Multi-View Reinforcement Learning

Cited by in corpus (1)

• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning