Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors
arXiv:2006.13205
Abstract
The ability to predict and plan into the future is fundamental for agents acting in the world. To reach a faraway goal, we predict trajectories at multiple timescales, first devising a coarse plan towards the goal and then gradually filling in details. In contrast, current learning approaches for visual prediction and planning fail on long-horizon tasks as they generate predictions (1) without considering goal information, and (2) at the finest temporal resolution, one step at a time. In this work we propose a framework for visual prediction and planning that is able to overcome both of these limitations. First, we formulate the problem of predicting towards a goal and propose the corresponding class of latent space goal-conditioned predictors (GCPs). GCPs significantly improve planning efficiency by constraining the search space to only those trajectories that reach the goal. Further, we show how GCPs can be naturally formulated as hierarchical models that, given two observations, predict an observation between them, and by recursively subdividing each part of the trajectory generate complete sequences. This divide-and-conquer strategy is effective at long-term prediction, and enables us to design an effective hierarchical planning algorithm that optimizes trajectories in a coarse-to-fine manner. We show that by using both goal-conditioning and hierarchical prediction, GCPs enable us to solve visual planning tasks with much longer horizon than previously possible.
arXiv commentProject page: orybkin.github.io/video-gcp. KP and OR contributed equally
References in corpus (27)
- Auto-Encoding Variational Bayes
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
- On the Variance of the Adaptive Learning Rate and Beyond
- Recurrent World Models Facilitate Policy Evolution
- Decomposing Motion and Content for Natural Video Sequence Prediction
- Learning Latent Dynamics for Planning from Pixels
- Stochastic Video Generation with a Learned Prior
- Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
- Stochastic Adversarial Video Prediction
- Towards Accurate Generative Models of Video: A New Metric & Challenges
- robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
- SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning
- One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning
- Hierarchical Long-term Video Prediction without Supervision
- Learning Plannable Representations with Causal InfoGAN
- Deep Dynamics Models for Learning Dexterous Manipulation
- Planning to Explore via Self-Supervised World Models
- Learning and Querying Fast Generative Models for Reinforcement Learning
- Simple and Effective VAE Training with Calibrated Decoders
- Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation
- Variational Temporal Abstraction
- Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation
- Improvisation through Physical Understanding: Using Novel Objects as Tools with Visual Foresight
- Keyframing the Future: Keyframe Discovery for Visual Prediction and Planning
- Divide-and-Conquer Monte Carlo Tree Search For Goal-Directed Planning
- Sub-Goal Trees -- a Framework for Goal-Based Reinforcement Learning
- Sub-Goal Trees -- a Framework for Goal-Directed Trajectory Prediction and Optimization