Hierarchical Planning for Long-Horizon Manipulation with Geometric and Symbolic Scene Graphs
arXiv:2012.07277
Abstract
We present a visually grounded hierarchical planning algorithm for long-horizon manipulation tasks. Our algorithm offers a joint framework of neuro-symbolic task planning and low-level motion generation conditioned on the specified goal. At the core of our approach is a two-level scene graph representation, namely geometric scene graph and symbolic scene graph. This hierarchical representation serves as a structured, object-centric abstraction of manipulation scenes. Our model uses graph neural networks to process these scene graphs for predicting high-level task plans and low-level motions. We demonstrate that our method scales to long-horizon tasks and generalizes well to novel task goals. We validate our method in a kitchen storage task in both physical simulation and the real world. Our experiments show that our method achieved over 70% success rate and nearly 90% of subgoal completion rate on the real robot while being four orders of magnitude faster in computation time compared to standard search-based task-and-motion planner.
arXiv commentAccepted to ICRA 2021
References in corpus (9)
- Learning Latent Dynamics for Planning from Pixels
- Discovering Symbolic Models from Deep Learning with Inductive Biases
- Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
- MONet: Unsupervised Scene Decomposition and Representation
- GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations
- Planning with Goal-Conditioned Policies
- Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors
- PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planning
- Regression Planning Networks