Hierarchical and Partially Observable Goal-driven Policy Learning with Goals Relational Graph

Abstract

We present a novel two-layer hierarchical reinforcement learning approach equipped with a Goals Relational Graph (GRG) for tackling the partially observable goal-driven task, such as goal-driven visual navigation. Our GRG captures the underlying relations of all goals in the goal space through a Dirichlet-categorical process that facilitates: 1) the high-level network raising a sub-goal towards achieving a designated final goal; 2) the low-level network towards an optimal policy; and 3) the overall system generalizing unseen environments and goals. We evaluate our approach with two settings of partially observable goal-driven tasks -- a grid-world domain and a robotic object search task. Our experimental results show that our approach exhibits superior generalization performance on both unseen environments and new goals.

arXiv commentCVPR2021

References in corpus (14)

• Continuous control with deep reinforcement learning
• Semi-Supervised Classification with Graph Convolutional Networks
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• On Evaluation of Embodied Navigation Agents
• AI2-THOR: An Interactive 3D Environment for Visual AI
• Building Generalizable Agents with a Realistic and Rich 3D Environment
• Data-Efficient Hierarchical Reinforcement Learning
• ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects
• Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model
• Planning with Goal-Conditioned Policies
• Semi-parametric Topological Memory for Navigation
• Visual Semantic Navigation using Scene Priors
• Learning hierarchical relationships for object-goal navigation
• Variational Recurrent Models for Solving Partially Observable Control Tasks