Hierarchical Reinforcement Learning Framework towards Multi-agent Navigation

Abstract

In this paper, we propose a navigation algorithm oriented to multi-agent environment. This algorithm is expressed as a hierarchical framework that contains a Hidden Markov Model (HMM) and a Deep Reinforcement Learning (DRL) structure. For simplification, we term our method Hierarchical Navigation Reinforcement Network (HNRN). In high- level architecture, we train an HMM to evaluate the agent's perception to obtain a score. According to this score, adaptive control action will be chosen. While in low-level architecture, two sub-systems are introduced, one is a differential target- driven system, which aims at heading to the target; the other is a collision avoidance DRL system, which is used for avoiding dynamic obstacles. The advantage of this hierarchical structure is decoupling the target-driven and collision avoidance tasks, leading to a faster and more stable model to be trained. The experiments indicate that our algorithm has higher learning efficiency and rate of success than traditional Velocity Obstacle (VO) algorithms or hybrid DRL method.

arXiv comment7 pages, 4 figures, submitted to ROBIO 2018

References in corpus (9)

• Social Force Model for Pedestrian Dynamics
• Playing Atari with Deep Reinforcement Learning
• From Perception to Decision: A Data-driven Approach to End-to-end Motion Planning for Autonomous Ground Robots
• Hindsight Experience Replay
• Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation
• Extracting Traffic Primitives Directly from Naturalistically Logged Data for Self-Driving Applications
• Socially Aware Motion Planning with Deep Reinforcement Learning
• Overview: A Hierarchical Framework for Plan Generation and Execution in Multi-Robot Systems
• Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments