Fully Distributed Multi-Robot Collision Avoidance via Deep Reinforcement Learning for Safe and Efficient Navigation in Complex Scenarios

Abstract

In this paper, we present a decentralized sensor-level collision avoidance policy for multi-robot systems, which shows promising results in practical applications. In particular, our policy directly maps raw sensor measurements to an agent's steering commands in terms of the movement velocity. As a first step toward reducing the performance gap between decentralized and centralized methods, we present a multi-scenario multi-stage training framework to learn an optimal policy. The policy is trained over a large number of robots in rich, complex environments simultaneously using a policy gradient based reinforcement learning algorithm. The learning algorithm is also integrated into a hybrid control framework to further improve the policy's robustness and effectiveness. We validate the learned sensor-level collision avoidance policy in a variety of simulated and real-world scenarios with thorough performance evaluations for large-scale multi-robot systems. The generalization of the learned policy is verified in a set of unseen scenarios including the navigation of a group of heterogeneous robots and a large-scale scenario with 100 robots. Although the policy is trained using simulation data only, we have successfully deployed it on physical robots with shapes and dynamics characteristics that are different from the simulated agents, in order to demonstrate the controller's robustness against the sim-to-real modeling error. Finally, we show that the collision-avoidance policy learned from multi-robot navigation tasks provides an excellent solution to the safe and effective autonomous navigation for a single robot working in a dense real human crowd. Our learned policy enables a robot to make effective progress in a crowd without getting stuck. Videos are available at https://sites.google.com/view/hybridmrca

References in corpus (2)

• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Socially Aware Motion Planning with Deep Reinforcement Learning

Cited by in corpus (18)

• UWB-Based Localization for Multi-UAV Systems and Collaborative Heterogeneous Multi-Robot Systems: a Survey
• COVID-Robot: Monitoring Social Distancing Constraints in Crowded Scenarios
• Cross-Modal Contrastive Learning of Representations for Navigation using Lightweight, Low-Cost Millimeter Wave Radar for Adverse Environmental Conditions
• Realtime Collision Avoidance for Mobile Robots in Dense Crowds using Implicit Multi-sensor Fusion and Deep Reinforcement Learning
• Ctrl-Z: Recovering from Instability in Reinforcement Learning
• Human-Inspired Multi-Agent Navigation using Knowledge Distillation
• Learning to Navigate in a VUCA Environment: Hierarchical Multi-expert Approach
• Learning World Transition Model for Socially Aware Robot Navigation
• Dynamically Feasible Deep Reinforcement Learning Policy for Robot Navigation in Dense Mobile Crowds
• Robot Inner Attention Modeling for Task-Adaptive Teaming of Heterogeneous Multi Robots
• Reciprocal Collision Avoidance for General Nonlinear Agents using Reinforcement Learning
• Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians
• Learning Resilient Behaviors for Navigation Under Uncertainty
• AirCapRL: Autonomous Aerial Human Motion Capture using Deep Reinforcement Learning
• OF-VO: Efficient Navigation among Pedestrians Using Commodity Sensors
• Socially-Aware Multi-Agent Following with 2D Laser Scans via Deep Reinforcement Learning and Potential Field
• DeepMNavigate: Deep Reinforced Multi-Robot Navigation Unifying Local & Global Collision Avoidance
• Move Beyond Trajectories: Distribution Space Coupling for Crowd Navigation