Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving

Abstract

Tactical decision making for autonomous driving is challenging due to the diversity of environments, the uncertainty in the sensor information, and the complex interaction with other road users. This paper introduces a general framework for tactical decision making, which combines the concepts of planning and learning, in the form of Monte Carlo tree search and deep reinforcement learning. The method is based on the AlphaGo Zero algorithm, which is extended to a domain with a continuous state space where self-play cannot be used. The framework is applied to two different highway driving cases in a simulated environment and it is shown to perform better than a commonly used baseline method. The strength of combining planning and learning is also illustrated by a comparison to using the Monte Carlo tree search or the neural network policy separately.

References in corpus (5)

• Continuous control with deep reinforcement learning
• Congested Traffic States in Empirical Observations and Microscopic Simulations
• An overview of gradient descent optimization algorithms
• Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
• Automated Speed and Lane Change Decision Making using Deep Reinforcement Learning

Cited by in corpus (26)

• Driving in Dense Traffic with Model-Free Reinforcement Learning
• Interaction-aware Decision-making for Automated Vehicles using Social Value Orientation
• Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Driving
• Car-Following Models: A Multidisciplinary Review
• Potential Game-Based Decision-Making for Autonomous Driving
• Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
• Resilient Branching MPC for Multi-Vehicle Traffic Scenarios Using Adversarial Disturbance Sequences
• A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learning
• Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applications
• Deep Reinforcement Learning and Transportation Research: A Comprehensive Review
• GIN: Graph-based Interaction-aware Constraint Policy Optimization for Autonomous Driving
• Decision-making for Autonomous Vehicles on Highway: Deep Reinforcement Learning with Continuous Action Horizon
• Decentralized Cooperative Lane Changing at Freeway Weaving Areas Using Multi-Agent Deep Reinforcement Learning
• Reinforcement Learning Based Safe Decision Making for Highway Autonomous Driving
• Decision-making Strategy on Highway for Autonomous Vehicles using Deep Reinforcement Learning
• Driving Tasks Transfer in Deep Reinforcement Learning for Decision-making of Autonomous Vehicles
• Off-Policy Risk-Sensitive Reinforcement Learning Based Constrained Robust Optimal Control
• An Online Spatial-Temporal Graph Trajectory Planner for Autonomous Vehicles
• Dueling Deep Q Network for Highway Decision Making in Autonomous Vehicles: A Case Study
• Decentralized Reinforcement Learning for Multi-Target Search and Detection by a Team of Drones
• Reinforcement Learning for Industrial Control Network Cyber Security Orchestration
• A Comparative Analysis of Deep Reinforcement Learning-enabled Freeway Decision-making for Automated Vehicles
• Automatically Learning Fallback Strategies with Model-Free Reinforcement Learning in Safety-Critical Driving Scenarios
• A New Approach for Tactical Decision Making in Lane Changing: Sample Efficient Deep Q Learning with a Safety Feedback Reward
• Goal-constrained Sparse Reinforcement Learning for End-to-End Driving
• A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning