Parameter Sharing Deep Deterministic Policy Gradient for Cooperative Multi-agent Reinforcement Learning

Abstract

Deep reinforcement learning for multi-agent cooperation and competition has been a hot topic recently. This paper focuses on cooperative multi-agent problem based on actor-critic methods under local observations settings. Multi agent deep deterministic policy gradient obtained state of art results for some multi-agent games, whereas, it cannot scale well with growing amount of agents. In order to boost scalability, we propose a parameter sharing deterministic policy gradient method with three variants based on neural networks, including actor-critic sharing, actor sharing and actor sharing with partially shared critic. Benchmarks from rllab show that the proposed method has advantages in learning speed and memory efficiency, well scales with growing amount of agents, and moreover, it can make full use of reward sharing and exchangeability if possible.

12 pages, 6 figures

References in corpus (3)

• Continuous control with deep reinforcement learning
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Hybrid Reward Architecture for Reinforcement Learning

Cited by in corpus (8)

• Modelling the Dynamic Joint Policy of Teammates with Attention Multi-agent DDPG
• Robust and Scalable Routing with Multi-Agent Deep Reinforcement Learning for MANETs
• Revisiting Parameter Sharing in Multi-Agent Deep Reinforcement Learning
• Learning Agent Communication under Limited Bandwidth by Message Pruning
• Parameter Sharing Reinforcement Learning Architecture for Multi Agent Driving Behaviors
• Event-Triggered Multi-agent Reinforcement Learning with Communication under Limited-bandwidth Constraint
• Coordination in Adversarial Sequential Team Games via Multi-Agent Deep Reinforcement Learning
• Reinforcement Learning Aided Sequential Optimization for Unsignalized Intersection Management of Robot Traffic