Collision Avoidance in Pedestrian-Rich Environments with Deep Reinforcement Learning

Abstract

Collision avoidance algorithms are essential for safe and efficient robot operation among pedestrians. This work proposes using deep reinforcement (RL) learning as a framework to model the complex interactions and cooperation with nearby, decision-making agents, such as pedestrians and other robots. Existing RL-based works assume homogeneity of agent properties, use specific motion models over short timescales, or lack a principled method to handle a large, possibly varying number of agents. Therefore, this work develops an algorithm that learns collision avoidance among a variety of heterogeneous, non-communicating, dynamic agents without assuming they follow any particular behavior rules. It extends our previous work by introducing a strategy using Long Short-Term Memory (LSTM) that enables the algorithm to use observations of an arbitrary number of other agents, instead of a small, fixed number of neighbors. The proposed algorithm is shown to outperform a classical collision avoidance algorithm, another deep RL-based algorithm, and scales with the number of agents better (fewer collisions, shorter time to goal) than our previously published learning-based approach. Analysis of the LSTM provides insights into how observations of nearby agents affect the hidden state and quantifies the performance impact of various agent ordering heuristics. The learned policy generalizes to several applications beyond the training scenarios: formation control (arrangement into letters), demonstrations on a fleet of four multirotors and on a fully autonomous robotic vehicle capable of traveling at human walking speed among pedestrians.

arXiv admin note: substantial text overlap with arXiv:1805.01956

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Sequence to Sequence Learning with Neural Networks
• TensorFlow: A system for large-scale machine learning
• End to End Learning for Self-Driving Cars
• Addressing Function Approximation Error in Actor-Critic Methods
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• Emergent Tool Use From Multi-Agent Autocurricula
• Dynamic Clustering via Asymptotics of the Dependent Dirichlet Process Mixture
• Dynamic Arrival Rate Estimation for Campus Mobility on Demand Network Graphs

Cited by in corpus (23)

• DRL-VO: Learning to Navigate Through Crowded Dynamic Scenes Using Velocity Obstacles
• Learning-Based UAV Path Planning for Data Collection with Integrated Collision Avoidance
• Interaction-aware Decision-making for Automated Vehicles using Social Value Orientation
• Learning Team-Based Navigation: A Review of Deep Reinforcement Learning Techniques for Multi-Agent Pathfinding
• Resilient Path Planning for UAVs in Data Collection under Adversarial Attacks
• IPAPRec: A promising tool for learning high-performance mapless navigation skills with deep reinforcement learning
• Improving robot navigation in crowded environments using intrinsic rewards
• SICNav: Safe and Interactive Crowd Navigation using Model Predictive Control and Bilevel Optimization
• Conformal Predictive Safety Filter for RL Controllers in Dynamic Environments
• A review of mobile robot motion planning methods: from classical motion planning workflows to reinforcement learning-based architectures
• Rethinking Trajectory Forecasting Evaluation
• Model-based Reinforcement Learning for Decentralized Multiagent Rendezvous
• Learning-Based UAV Trajectory Optimization with Collision Avoidance and Connectivity Constraints
• SHINE: Social Homology Identification for Navigation in Crowded Environments
• AVOCADO: Adaptive Optimal Collision Avoidance driven by Opinion
• Pedestrian Collision Avoidance for Autonomous Vehicles at Unsignalized Intersection Using Deep Q-Network
• Dynamically Feasible Deep Reinforcement Learning Policy for Robot Navigation in Dense Mobile Crowds
• Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians
• Safe Deep Q-Network for Autonomous Vehicles at Unsignalized Intersection
• Lyapunov-Based Reinforcement Learning for Decentralized Multi-Agent Control
• CosIn: A Statistical-based Algorithm for Computation of Space-speed Time Delay in Pedestrian Motion
• Prediction-Based Reachability for Collision Avoidance in Autonomous Driving
• GIANT - Global Path Integration and Attentive Graph Networks for Multi-Agent Trajectory Planning