Learning Transferable Concepts in Deep Reinforcement Learning

Abstract

While humans and animals learn incrementally during their lifetimes and exploit their experience to solve new tasks, standard deep reinforcement learning methods specialize to solve only one task at a time. As a result, the information they acquire is hardly reusable in new situations. Here, we introduce a new perspective on the problem of leveraging prior knowledge to solve future tasks. We show that learning discrete representations of sensory inputs can provide a high-level abstraction that is common across multiple tasks, thus facilitating the transference of information. In particular, we show that it is possible to learn such representations by self-supervision, following an information theoretic approach. Our method is able to learn concepts in locomotive and optimal control tasks that increase the sample efficiency in both known and unknown tasks, opening a new path to endow artificial agents with generalization abilities.

References in corpus (23)

• Trust Region Policy Optimization
• The information bottleneck method
• Soft Actor-Critic Algorithms and Applications
• Dueling Network Architectures for Deep Reinforcement Learning
• Learning deep representations by mutual information estimation and maximization
• Robots that can adapt like animals
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• Emergence of Locomotion Behaviours in Rich Environments
• Noisy Networks for Exploration
• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• Unsupervised Learning of Object Keypoints for Perception and Control
• Learning Independent Causal Mechanisms
• Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck
• Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?
• Investigating Generalisation in Continuous Deep Reinforcement Learning
• Entity Abstraction in Visual Model-Based Reinforcement Learning
• Action and Perception as Divergence Minimization
• On Inductive Biases in Deep Reinforcement Learning
• Hierarchical Reinforcement Learning via Advantage-Weighted Information Maximization
• OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning
• Efficient human-like semantic representations via the Information Bottleneck principle
• Investigating Simple Object Representations in Model-Free Deep Reinforcement Learning
• Learning State Abstractions for Transfer in Continuous Control