Deep Reinforcement Learning and Transportation Research: A Comprehensive Review

Abstract

Deep reinforcement learning (DRL) is an emerging methodology that is transforming the way many complicated transportation decision-making problems are tackled. Researchers have been increasingly turning to this powerful learning-based methodology to solve challenging problems across transportation fields. While many promising applications have been reported in the literature, there remains a lack of comprehensive synthesis of the many DRL algorithms and their uses and adaptations. The objective of this paper is to fill this gap by conducting a comprehensive, synthesized review of DRL applications in transportation. We start by offering an overview of the DRL mathematical background, popular and promising DRL algorithms, and some highly effective DRL extensions. Building on this overview, a systematic investigation of about 150 DRL studies that have appeared in the transportation literature, divided into seven different categories, is performed. Building on this review, we continue to examine the applicability, strengths, shortcomings, and common and application-specific issues of DRL techniques with regard to their applications in transportation. In the end, we recommend directions for future research and present available resources for actually implementing DRL.

References in corpus (47)

• Continuous control with deep reinforcement learning
• Playing Atari with Deep Reinforcement Learning
• Trust Region Policy Optimization
• Prioritized Experience Replay
• Dueling Network Architectures for Deep Reinforcement Learning
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Learning Combinatorial Optimization Algorithms over Graphs
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Deep Reinforcement Learning: An Overview
• Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving
• Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
• Continuous Deep Q-Learning with Model-based Acceleration
• Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving
• Deep Q-learning from Demonstrations
• Maximum Entropy Deep Inverse Reinforcement Learning
• FeUdal Networks for Hierarchical Reinforcement Learning
• Using a Deep Reinforcement Learning Agent for Traffic Signal Control
• Memory-based control with recurrent neural networks
• Hierarchical Reinforcement Learning for Self-Driving Decision-Making without Reliance on Labeled Driving Data
• DeepPool: Distributed Model-free Algorithm for Ride-sharing using Deep Reinforcement Learning
• Deep Q-Learning for Same-Day Delivery with Vehicles and Drones
• Adaptive Traffic Signal Control: Deep Reinforcement Learning Algorithm with Experience Replay and Target Network
• Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge
• Third-Person Imitation Learning
• End-to-End Deep Reinforcement Learning for Lane Keeping Assist
• Controlling an Autonomous Vehicle with Deep Reinforcement Learning
• Strategic Attentive Writer for Learning Macro-Actions
• Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networks
• Acme: A Research Framework for Distributed Reinforcement Learning
• Adaptive Behavior Generation for Autonomous Driving using Deep Reinforcement Learning with Compact Semantic States
• Freeway Merging in Congested Traffic based on Multipolicy Decision Making with Passive Actor Critic
• rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch
• Deep Reinforcement Learning for Autonomous Driving
• Longitudinal Dynamic versus Kinematic Models for Car-Following Control Using Deep Reinforcement Learning
• Integrating independent and centralized multi-agent reinforcement learning for traffic signal network optimization
• ORL: Reinforcement Learning Benchmarks for Online Stochastic Optimization Problems
• Cooperative Lane Changing via Deep Reinforcement Learning
• Autonomous Ramp Merge Maneuver Based on Reinforcement Learning with Continuous Action Space
• Automated Lane Change Strategy using Proximal Policy Optimization-based Deep Reinforcement Learning
• Differential Variable Speed Limits Control for Freeway Recurrent Bottlenecks via Deep Reinforcement learning
• Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach
• Deep Reinforcement Learning Based High-level Driving Behavior Decision-making Model in Heterogeneous Traffic
• MOVI: A Model-Free Approach to Dynamic Fleet Management
• Deep Reinforcement Learning for Multi-Driver Vehicle Dispatching and Repositioning Problem
• Smart Train Operation Algorithms based on Expert Knowledge and Reinforcement Learning
• Gamifying the Vehicle Routing Problem with Stochastic Requests