Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning

Abstract

Electric Vehicle (EV) has become a preferable choice in the modern transportation system due to its environmental and energy sustainability. However, in many large cities, EV drivers often fail to find the proper spots for charging, because of the limited charging infrastructures and the spatiotemporally unbalanced charging demands. Indeed, the recent emergence of deep reinforcement learning provides great potential to improve the charging experience from various aspects over a long-term horizon. In this paper, we propose a framework, named Multi-Agent Spatio-Temporal Reinforcement Learning (Master), for intelligently recommending public accessible charging stations by jointly considering various long-term spatiotemporal factors. Specifically, by regarding each charging station as an individual agent, we formulate this problem as a multi-objective multi-agent reinforcement learning task. We first develop a multi-agent actor-critic framework with the centralized attentive critic to coordinate the recommendation between geo-distributed agents. Moreover, to quantify the influence of future potential charging competition, we introduce a delayed access strategy to exploit the knowledge of future charging competition during training. After that, to effectively optimize multiple learning objectives, we extend the centralized attentive critic to multi-critics and develop a dynamic gradient re-weighting strategy to adaptively guide the optimization direction. Finally, extensive experiments on two real-world datasets demonstrate that Master achieves the best comprehensive performance compared with nine baseline approaches.

12 pages, 10 figures

References in corpus (13)

• Continuous control with deep reinforcement learning
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• CoLight: Learning Network-level Cooperation for Traffic Signal Control
• Counterfactual Multi-Agent Policy Gradients
• Learning Attentional Communication for Multi-Agent Cooperation
• Actor-Attention-Critic for Multi-Agent Reinforcement Learning
• Learning Multiagent Communication with Backpropagation
• Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
• Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
• CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms
• Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching
• Out-of-Town Recommendation with Travel Intention Modeling

Cited by in corpus (9)

• A Survey on Reinforcement Learning for Recommender Systems
• Reinforcement Learning-enhanced Shared-account Cross-domain Sequential Recommendation
• MugRep: A Multi-Task Hierarchical Graph Representation Learning Framework for Real Estate Appraisal
• Reinforcement Learning-based Placement of Charging Stations in Urban Road Networks
• A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions
• A privacy preserving querying mechanism with high utility for electric vehicles
• Agent-Based Decentralized Energy Management of EV Charging Station with Solar Photovoltaics via Multi-Agent Reinforcement Learning
• A Preference-aware Meta-optimization Framework for Personalized Vehicle Energy Consumption Estimation
• Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysis