Multi-agent Reinforcement Learning for Networked System Control

Abstract

This paper considers multi-agent reinforcement learning (MARL) in networked system control. Specifically, each agent learns a decentralized control policy based on local observations and messages from connected neighbors. We formulate such a networked MARL (NMARL) problem as a spatiotemporal Markov decision process and introduce a spatial discount factor to stabilize the training of each local agent. Further, we propose a new differentiable communication protocol, called NeurComm, to reduce information loss and non-stationarity in NMARL. Based on experiments in realistic NMARL scenarios of adaptive traffic signal control and cooperative adaptive cruise control, an appropriate spatial discount factor effectively enhances the learning curves of non-communicative MARL algorithms, while NeurComm outperforms existing communication protocols in both learning efficiency and control performance.

ICLR 2020

References in corpus (5)

• Continuous control with deep reinforcement learning
• Counterfactual Multi-Agent Policy Gradients
• Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
• Revisiting the Master-Slave Architecture in Multi-Agent Deep Reinforcement Learning
• Value Propagation for Decentralized Networked Deep Multi-agent Reinforcement Learning

Cited by in corpus (9)

• Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic
• OnSlicing: Online End-to-End Network Slicing with Reinforcement Learning
• Multi-Agent Reinforcement Learning Based on Representational Communication for Large-Scale Traffic Signal Control
• Graph Convolutional Value Decomposition in Multi-Agent Reinforcement Learning
• A Decentralized Policy Gradient Approach to Multi-task Reinforcement Learning
• PowerNet: Multi-agent Deep Reinforcement Learning for Scalable Powergrid Control
• A game-theoretic analysis of networked system control for common-pool resource management using multi-agent reinforcement learning
• Multi-Agent Intention Sharing via Leader-Follower Forest
• Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problem