Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images

Abstract

We introduce Embed to Control (E2C), a method for model learning and control of non-linear dynamical systems from raw pixel images. E2C consists of a deep generative model, belonging to the family of variational autoencoders, that learns to generate image trajectories from a latent space in which the dynamics is constrained to be locally linear. Our model is derived directly from an optimal control formulation in latent space, supports long-term prediction of image sequences and exhibits strong performance on a variety of complex control problems.

Final NIPS version

References in corpus (4)

• End-to-End Training of Deep Visuomotor Policies
• NICE: Non-linear Independent Components Estimation
• DRAW: A Recurrent Neural Network For Image Generation
• Learning Stochastic Recurrent Networks

Cited by in corpus (28)

• A Brief Survey of Deep Reinforcement Learning
• An Introduction to Variational Autoencoders
• Dueling Network Architectures for Deep Reinforcement Learning
• State Representation Learning for Control: An Overview
• Dynamical Variational Autoencoders: A Comprehensive Review
• Learning Plannable Representations with Causal InfoGAN
• Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning
• Online Service Migration in Mobile Edge with Incomplete System Information: A Deep Recurrent Actor-Critic Learning Approach
• Optimization of Pressure Management Strategies for Geological CO2 Sequestration Using Surrogate Model-based Reinforcement Learning
• Representation Matters: Offline Pretraining for Sequential Decision Making
• Model predictive control for robust quantum state preparation
• Integrating Contrastive Learning with Dynamic Models for Reinforcement Learning from Images
• Reaching Through Latent Space: From Joint Statistics to Path Planning in Manipulation
• Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills
• Boredom-driven curious learning by Homeo-Heterostatic Value Gradients
• Learning to Optimize in Model Predictive Control
• Adaptive Path-Integral Autoencoder: Representation Learning and Planning for Dynamical Systems
• Learning State Conditioned Linear Mappings for Low-Dimensional Control of Robotic Manipulators
• Block Contextual MDPs for Continual Learning
• A Free Lunch from the Noise: Provable and Practical Exploration for Representation Learning
• Fractional Transfer Learning for Deep Model-Based Reinforcement Learning
• $\mathtt{MORALS}$: Analysis of High-Dimensional Robot Controllers via Topological Tools in a Latent Space
• Learning Representations for Pixel-based Control: What Matters and Why?
• Interleaved Multitask Learning with Energy Modulated Learning Progress
• Operator Shifting for Model-based Policy Evaluation
• Dynamic Variational Autoencoders for Visual Process Modeling
• High Performance Across Two Atari Paddle Games Using the Same Perceptual Control Architecture Without Training
• GEM: Group Enhanced Model for Learning Dynamical Control Systems