Self-supervised Learning of Image Embedding for Continuous Control

Abstract

Operating directly from raw high dimensional sensory inputs like images is still a challenge for robotic control. Recently, Reinforcement Learning methods have been proposed to solve specific tasks end-to-end, from pixels to torques. However, these approaches assume the access to a specified reward which may require specialized instrumentation of the environment. Furthermore, the obtained policy and representations tend to be task specific and may not transfer well. In this work we investigate completely self-supervised learning of a general image embedding and control primitives, based on finding the shortest time to reach any state. We also introduce a new structure for the state-action value function that builds a connection between model-free and model-based methods, and improves the performance of the learning algorithm. We experimentally demonstrate these findings in three simulated robotic tasks.

Contributed talk at Inference to Control workshop at NeurIPS2018

References in corpus (4)

• Emergence of Locomotion Behaviours in Rich Environments
• Visual Reinforcement Learning with Imagined Goals
• Stochastic Adversarial Video Prediction
• Successor Features for Transfer in Reinforcement Learning

Cited by in corpus (17)

• A survey on intrinsic motivation in reinforcement learning
• SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation
• Skew-Fit: State-Covering Self-Supervised Reinforcement Learning
• Search on the Replay Buffer: Bridging Planning and Reinforcement Learning
• Goal-conditioned Imitation Learning
• Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learning
• Model-Based Visual Planning with Self-Supervised Functional Distances
• Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skills
• Contextual Imagined Goals for Self-Supervised Robotic Learning
• Sub-policy Adaptation for Hierarchical Reinforcement Learning
• Plan2Vec: Unsupervised Representation Learning by Latent Plans
• Weakly-Supervised Reinforcement Learning for Controllable Behavior
• Language Conditioned Imitation Learning over Unstructured Data
• Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discovery
• MAME : Model-Agnostic Meta-Exploration
• Wish you were here: Hindsight Goal Selection for long-horizon dexterous manipulation
• Multi-task Reinforcement Learning with a Planning Quasi-Metric