Unsupervised state representation learning with robotic priors: a robustness benchmark

Abstract

Our understanding of the world depends highly on our capacity to produce intuitive and simplified representations which can be easily used to solve problems. We reproduce this simplification process using a neural network to build a low dimensional state representation of the world from images acquired by a robot. As in Jonschkowski et al. 2015, we learn in an unsupervised way using prior knowledge about the world as loss functions called robotic priors and extend this approach to high dimension richer images to learn a 3D representation of the hand position of a robot from RGB images. We propose a quantitative evaluation of the learned representation using nearest neighbors in the state space that allows to assess its quality and show both the potential and limitations of robotic priors in realistic environments. We augment image size, add distractors and domain randomization, all crucial components to achieve transfer learning to real robots. Finally, we also contribute a new prior to improve the robustness of the representation. The applications of such low dimensional state representation range from easing reinforcement learning (RL) and knowledge transfer across tasks, to facilitating learning from raw data with more efficient and compact high level representations. The results show that the robotic prior approach is able to extract high level representation as the 3D position of an arm and organize it into a compact and coherent space of states in a challenging dataset.

arXiv commentICRA 2018 submission

References in corpus (3)

• Deep Residual Learning for Image Recognition
• Predictive State Representations: A New Theory for Modeling Dynamical Systems
• A new embedding quality assessment method for manifold learning

Cited by in corpus (8)

• State Representation Learning for Control: An Overview
• An Empirical Analysis of Deep Learning for Cardinality Estimation
• Learning Actionable Representations from Visual Observations
• Direct and inverse modeling of soft robots by learning a condensed FEM model
• Low Dimensional State Representation Learning with Reward-shaped Priors
• Learning State Representations in Complex Systems with Multimodal Data
• Defining the problem of Observation Learning
• Autonomous Learning of Features for Control: Experiments with Embodied and Situated Agents