Independently Controllable Factors
arXiv:1708.01289
Abstract
It has been postulated that a good representation is one that disentangles the underlying explanatory factors of variation. However, it remains an open question what kind of training framework could potentially achieve that. Whereas most previous work focuses on the static setting (e.g., with images), we postulate that some of the causal factors could be discovered if the learner is allowed to interact with its environment. The agent can experiment with different actions and observe their effects. More specifically, we hypothesize that some of these factors correspond to aspects of the environment which are independently controllable, i.e., that there exists a policy and a learnable feature for each such aspect of the environment, such that this policy can yield changes in that feature with minimal changes to other features that explain the statistical variations in the observed data. We propose a specific objective function to find such factors and verify experimentally that it can indeed disentangle independently controllable aspects of the environment without any extrinsic reward signal.
References in corpus (2)
Cited by in corpus (30)
- State Representation Learning for Control: An Overview
- Measuring the tendency of CNNs to Learn Surface Statistical Regularities
- Isolating Sources of Disentanglement in Variational Autoencoders
- A survey on intrinsic motivation in reinforcement learning
- Curiosity Driven Exploration of Learned Disentangled Goal Spaces
- Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Exploration
- On the Binding Problem in Artificial Neural Networks
- Composable Planning with Attributes
- Auto-Encoding Total Correlation Explanation
- Unsupervised State Representation Learning in Atari
- Desiderata for Representation Learning: A Causal Perspective
- Symmetry-Based Disentangled Representation Learning requires Interaction with Environments
- Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems
- Task-Agnostic Dynamics Priors for Deep Reinforcement Learning
- Learning Group Structure and Disentangled Representations of Dynamical Environments
- Learning to Plan Hierarchically from Curriculum
- Disentangling Controllable and Uncontrollable Factors of Variation by Interacting with the World
- Which Mutual-Information Representation Learning Objectives are Sufficient for Control?
- Deep Learning in Information Security
- Weakly-Supervised Reinforcement Learning for Controllable Behavior
- Action-Sufficient State Representation Learning for Control with Structural Constraints
- Learning Independently-Obtainable Reward Functions
- Affordance as general value function: A computational model
- Unsupervised Emergence of Spatial Structure from Sensorimotor Prediction
- Linear Disentangled Representations and Unsupervised Action Estimation
- On the Sensory Commutativity of Action Sequences for Embodied Agents
- Disentangled (Un)Controllable Features
- Disentangling Video with Independent Prediction
- Towards intervention-centric causal reasoning in learning agents
- Learning in Sparse Rewards settings through Quality-Diversity algorithms