Independently Controllable Features

Abstract

Finding features that disentangle the different causes of variation in real data is a difficult task, that has nonetheless received considerable attention in static domains like natural images. Interactive environments, in which an agent can deliberately take actions, offer an opportunity to tackle this task better, because the agent can experiment with different actions and observe their effects. We introduce the idea that in interactive environments, latent factors that control the variation in observed data can be identified by figuring out what the agent can control. We propose a naive method to find factors that explain or measure the effect of the actions of a learner, and test it in illustrative experiments.

RLDM submission

References in corpus (2)

• Action-Conditional Video Prediction using Deep Networks in Atari Games
• Deep Successor Reinforcement Learning

Cited by in corpus (23)

• Causality for Machine Learning
• Learning Disentangled Representations in the Imaging Domain
• Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning
• A survey on intrinsic motivation in reinforcement learning
• Contingency-Aware Exploration in Reinforcement Learning
• Auto-Encoding Total Correlation Explanation
• Operationally meaningful representations of physical systems in neural networks
• Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness
• Integrating Contrastive Learning with Dynamic Models for Reinforcement Learning from Images
• Symmetry-Based Disentangled Representation Learning requires Interaction with Environments
• Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems
• Learning to Plan Hierarchically from Curriculum
• Disentangling Controllable and Uncontrollable Factors of Variation by Interacting with the World
• Weakly-Supervised Reinforcement Learning for Controllable Behavior
• Unsupervised Emergence of Egocentric Spatial Structure from Sensorimotor Prediction
• Simple Sensor Intentions for Exploration
• Exploration via Hindsight Goal Generation
• Learning Independently-Obtainable Reward Functions
• Disentangling Controllable Object through Video Prediction Improves Visual Reinforcement Learning
• Speeding up reinforcement learning by combining attention and agency features
• On the Sensory Commutativity of Action Sequences for Embodied Agents
• Label-Based Diversity Measure Among Hidden Units of Deep Neural Networks: A Regularization Method
• Meaning Versus Information, Prediction Versus Memory, and Question Versus Answer