Broadly-Exploring, Local-Policy Trees for Long-Horizon Task Planning

Abstract

Long-horizon planning in realistic environments requires the ability to reason over sequential tasks in high-dimensional state spaces with complex dynamics. Classical motion planning algorithms, such as rapidly-exploring random trees, are capable of efficiently exploring large state spaces and computing long-horizon, sequential plans. However, these algorithms are generally challenged with complex, stochastic, and high-dimensional state spaces as well as in the presence of narrow passages, which naturally emerge in tasks that interact with the environment. Machine learning offers a promising solution for its ability to learn general policies that can handle complex interactions and high-dimensional observations. However, these policies are generally limited in horizon length. Our approach, Broadly-Exploring, Local-policy Trees (BELT), merges these two approaches to leverage the strengths of both through a task-conditioned, model-based tree search. BELT uses an RRT-inspired tree search to efficiently explore the state space. Locally, the exploration is guided by a task-conditioned, learned policy capable of performing general short-horizon tasks. This task space can be quite general and abstract; its only requirements are to be sampleable and to well-cover the space of useful tasks. This search is aided by a task-conditioned model that temporally extends dynamics propagation to allow long-horizon search and sequential reasoning over tasks. BELT is demonstrated experimentally to be able to plan long-horizon, sequential trajectories with a goal conditioned policy and generate plans that are robust.

References in corpus (10)

• SHOP2: An HTN Planning System
• Model-Based Reinforcement Learning for Atari
• Learning Latent Dynamics for Planning from Pixels
• Visual Reinforcement Learning with Imagined Goals
• Data-Efficient Hierarchical Reinforcement Learning
• Multi-Level Discovery of Deep Options
• Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?
• Composable Planning with Attributes
• Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments
• Prediction, Consistency, Curvature: Representation Learning for Locally-Linear Control

Cited by in corpus (5)

• Learning Robotic Navigation from Experience: Principles, Methods, and Recent Results
• A Survey on the Integration of Machine Learning with Sampling-based Motion Planning
• Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills
• Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks
• A Simple Approach to Continual Learning by Transferring Skill Parameters