Drop-Bottleneck: Learning Discrete Compressed Representation for Noise-Robust Exploration

Abstract

We propose a novel information bottleneck (IB) method named Drop-Bottleneck, which discretely drops features that are irrelevant to the target variable. Drop-Bottleneck not only enjoys a simple and tractable compression objective but also additionally provides a deterministic compressed representation of the input variable, which is useful for inference tasks that require consistent representation. Moreover, it can jointly learn a feature extractor and select features considering each feature dimension's relevance to the target task, which is unattainable by most neural network-based IB methods. We propose an exploration method based on Drop-Bottleneck for reinforcement learning tasks. In a multitude of noisy and reward sparse maze navigation tasks in VizDoom (Kempka et al., 2016) and DMLab (Beattie et al., 2016), our exploration method achieves state-of-the-art performance. As a new IB framework, we demonstrate that Drop-Bottleneck outperforms Variational Information Bottleneck (VIB) (Alemi et al., 2017) in multiple aspects including adversarial robustness and dimensionality reduction.

Accepted to ICLR 2021. Code at http://vision.snu.ac.kr/projects/db

References in corpus (10)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• Estimating Mutual Information
• Categorical Reparameterization with Gumbel-Softmax
• The information bottleneck method
• Learning deep representations by mutual information estimation and maximization
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle
• Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck
• Restricting the Flow: Information Bottlenecks for Attribution