Deep Visual Foresight for Planning Robot Motion

Abstract

A key challenge in scaling up robot learning to many skills and environments is removing the need for human supervision, so that robots can collect their own data and improve their own performance without being limited by the cost of requesting human feedback. Model-based reinforcement learning holds the promise of enabling an agent to learn to predict the effects of its actions, which could provide flexible predictive models for a wide range of tasks and environments, without detailed human supervision. We develop a method for combining deep action-conditioned video prediction models with model-predictive control that uses entirely unlabeled training data. Our approach does not require a calibrated camera, an instrumented training set-up, nor precise sensing and actuation. Our results show that our method enables a real robot to perform nonprehensile manipulation -- pushing objects -- and can handle novel objects not seen during training.

ICRA 2017. Supplementary video: https://sites.google.com/site/robotforesight/

References in corpus (4)

• Auto-Encoding Variational Bayes
• Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
• Unsupervised Learning for Physical Interaction through Video Prediction
• Learning to Poke by Poking: Experiential Learning of Intuitive Physics

Cited by in corpus (14)

• NIPS 2016 Tutorial: Generative Adversarial Networks
• Deep Reinforcement Learning: An Overview
• Visual Reinforcement Learning with Imagined Goals
• MagNet: a Two-Pronged Defense against Adversarial Examples
• A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
• Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight
• Learning Latent Plans from Play
• Prediction and Control with Temporal Segment Models
• Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation
• Transformation-based Adversarial Video Prediction on Large-Scale Data
• Time Reversal as Self-Supervision
• 3D-OES: Viewpoint-Invariant Object-Factorized Environment Simulators
• Motion2Vec: Semi-Supervised Representation Learning from Surgical Videos
• Meaning Versus Information, Prediction Versus Memory, and Question Versus Answer