Decentralized Cooperative Planning for Automated Vehicles with Hierarchical Monte Carlo Tree Search

Abstract

Today's automated vehicles lack the ability to cooperate implicitly with others. This work presents a Monte Carlo Tree Search (MCTS) based approach for decentralized cooperative planning using macro-actions for automated vehicles in heterogeneous environments. Based on cooperative modeling of other agents and Decoupled-UCT (a variant of MCTS), the algorithm evaluates the state-action-values of each agent in a cooperative and decentralized manner, explicitly modeling the interdependence of actions between traffic participants. Macro-actions allow for temporal extension over multiple time steps and increase the effective search depth requiring fewer iterations to plan over longer horizons. Without predefined policies for macro-actions, the algorithm simultaneously learns policies over and within macro-actions. The proposed method is evaluated under several conflict scenarios, showing that the algorithm can achieve effective cooperative planning with learned macro-actions in heterogeneous environments.

References in corpus (1)

• Learning for Multi-robot Cooperation in Partially Observable Stochastic Environments with Macro-actions

Cited by in corpus (9)

• Monte Carlo Tree Search: A Review of Recent Modifications and Applications
• Addressing Inherent Uncertainty: Risk-Sensitive Behavior Generation for Automated Driving using Distributional Reinforcement Learning
• Decentralized Cooperative Planning for Automated Vehicles with Continuous Monte Carlo Tree Search
• Automatic Intersection Management in Mixed Traffic Using Reinforcement Learning and Graph Neural Networks
• Accelerating Cooperative Planning for Automated Vehicles with Learned Heuristics and Monte Carlo Tree Search
• Optimizing Delegation in Collaborative Human-AI Hybrid Teams
• A Model for Multi-Agent Autonomy That Uses Opinion Dynamics and Multi-Objective Behavior Optimization
• Optimizing Risk-averse Human-AI Hybrid Teams
• Fast Long-Term Multi-Scenario Prediction for Maneuver Planning at Unsignalized Intersections