Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey

Abstract

Recent technology development brings the boom of numerous new Demand-Driven Services (DDS) into urban lives, including ridesharing, on-demand delivery, express systems and warehousing. In DDS, a service loop is an elemental structure, including its service worker, the service providers and corresponding service targets. The service workers should transport either people or parcels from the providers to the target locations. Various planning tasks within DDS can thus be classified into two individual stages: 1) Dispatching, which is to form service loops from demand/supply distributions, and 2) Routing, which is to decide specific serving orders within the constructed loops. Generating high-quality strategies in both stages is important to develop DDS but faces several challenges. Meanwhile, deep reinforcement learning (DRL) has been developed rapidly in recent years. It is a powerful tool to solve these problems since DRL can learn a parametric model without relying on too many problem-based assumptions and optimize long-term effects by learning sequential decisions. In this survey, we first define DDS, then highlight common applications and important decision/control problems within. For each problem, we comprehensively introduce the existing DRL solutions. We also introduce open simulation environments for development and evaluation of DDS applications. Finally, we analyze remaining challenges and discuss further research opportunities in DRL solutions for DDS.

41 pages. survey preprint

References in corpus (17)

• Continuous control with deep reinforcement learning
• Semi-Supervised Classification with Graph Convolutional Networks
• Playing Atari with Deep Reinforcement Learning
• Trust Region Policy Optimization
• Dueling Network Architectures for Deep Reinforcement Learning
• Mean Field Multi-Agent Reinforcement Learning
• Sample Efficient Actor-Critic with Experience Replay
• Deep Reinforcement Learning for Electric Vehicle Routing Problem with Time Windows
• Meta-Reinforcement Learning of Structured Exploration Strategies
• DeepPool: Distributed Model-free Algorithm for Ride-sharing using Deep Reinforcement Learning
• Heterogeneous Attentions for Solving Pickup and Delivery Problem via Deep Reinforcement Learning
• Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks
• Reward Design for Driver Repositioning Using Multi-Agent Reinforcement Learning
• Reinforcement Learning with Convex Constraints
• Reinforcement Learning with Combinatorial Actions: An Application to Vehicle Routing
• Learning to Solve Vehicle Routing Problems with Time Windows through Joint Attention
• Multi-Decoder Attention Model with Embedding Glimpse for Solving Vehicle Routing Problems