Learning Awareness Models

Abstract

We consider the setting of an agent with a fixed body interacting with an unknown and uncertain external world. We show that models trained to predict proprioceptive information about the agent's body come to represent objects in the external world. In spite of being trained with only internally available signals, these dynamic body models come to represent external objects through the necessity of predicting their effects on the agent's own body. That is, the model learns holistic persistent representations of objects in the world, even though the only training signals are body signals. Our dynamics model is able to successfully predict distributions over 132 sensor readings over 100 steps into the future and we demonstrate that even when the body is no longer in contact with an object, the latent variables of the dynamics model continue to represent its shape. We show that active data collection by maximizing the entropy of predictions about the body---touch sensors, proprioception and vestibular information---leads to learning of dynamic models that show superior performance when used for control. We also collect data from a real robotic hand and show that the same models can be used to answer questions about properties of objects in the real world. Videos with qualitative results of our models are available at https://goo.gl/mZuqAV.

arXiv commentAccepted to ICLR 2018

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• End to End Learning for Self-Driving Cars
• Emergence of Locomotion Behaviours in Rich Environments
• Distributed Prioritized Experience Replay
• Parameter Space Noise for Exploration
• Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning
• Learning Stochastic Recurrent Networks
• Sequential Neural Models with Stochastic Layers
• A Bayesian Sampling Approach to Exploration in Reinforcement Learning
• Deep Kalman Filters
• The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?
• Black box variational inference for state space models
• The Consciousness Prior
• Curiosity-driven reinforcement learning with homeostatic regulation
• Predictive-State Decoders: Encoding the Future into Recurrent Networks
• Unsupervised preprocessing for Tactile Data

Cited by in corpus (9)

• Graph networks as learnable physics engines for inference and control
• dm_control: Software and Tasks for Continuous Control
• Neural Predictive Belief Representations
• Shaping Belief States with Generative Environment Models for RL
• Temporal Difference Variational Auto-Encoder
• Learning to Plan Optimistically: Uncertainty-Guided Deep Exploration via Latent Model Ensembles
• Model-based Meta Reinforcement Learning using Graph Structured Surrogate Models
• Swoosh! Rattle! Thump! -- Actions that Sound
• Temporally Abstract Partial Models