Data-Efficient Learning of Feedback Policies from Image Pixels using Deep Dynamical Models

Abstract

Data-efficient reinforcement learning (RL) in continuous state-action spaces using very high-dimensional observations remains a key challenge in developing fully autonomous systems. We consider a particularly important instance of this challenge, the pixels-to-torques problem, where an RL agent learns a closed-loop control policy ("torques") from pixel information only. We introduce a data-efficient, model-based reinforcement learning algorithm that learns such a closed-loop policy directly from pixel information. The key ingredient is a deep dynamical model for learning a low-dimensional feature embedding of images jointly with a predictive model in this low-dimensional feature space. Joint learning is crucial for long-term predictions, which lie at the core of the adaptive nonlinear model predictive control strategy that we use for closed-loop control. Compared to state-of-the-art RL methods for continuous states and actions, our approach learns quickly, scales to high-dimensional state spaces, is lightweight and an important step toward fully autonomous end-to-end learning from pixels to torques.

References in corpus (1)

• Deep Learning in Neural Networks: An Overview

Cited by in corpus (9)

• State Representation Learning for Control: An Overview
• Towards Deep Symbolic Reinforcement Learning
• Multi-Objective Deep Reinforcement Learning
• Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks
• Learning to Perform Physics Experiments via Deep Reinforcement Learning
• Generative Temporal Models with Memory
• Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks
• Exploratory State Representation Learning
• State Representation Learning from Demonstration