R-MADDPG for Partially Observable Environments and Limited Communication

Abstract

There are several real-world tasks that would benefit from applying multiagent reinforcement learning (MARL) algorithms, including the coordination among self-driving cars. The real world has challenging conditions for multiagent learning systems, such as its partial observable and nonstationary nature. Moreover, if agents must share a limited resource (e.g. network bandwidth) they must all learn how to coordinate resource use. This paper introduces a deep recurrent multiagent actor-critic framework (R-MADDPG) for handling multiagent coordination under partial observable set-tings and limited communication. We investigate recurrency effects on performance and communication use of a team of agents. We demonstrate that the resulting framework learns time dependencies for sharing missing observations, handling resource limitations, and developing different communication patterns among agents.

arXiv commentReinforcement Learning for Real Life (RL4RealLife) Workshop in the 36th International Conference on Machine Learning, Long Beach, California, USA, 2019

References in corpus (10)

• Prioritized Experience Replay
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• Counterfactual Multi-Agent Policy Gradients
• Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
• A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
• Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability
• Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks
• Revisiting the Master-Slave Architecture in Multi-Agent Deep Reinforcement Learning
• Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks
• Scalable Centralized Deep Multi-Agent Reinforcement Learning via Policy Gradients

Cited by in corpus (11)

• Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence
• Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning
• FACMAC: Factored Multi-Agent Centralised Policy Gradients
• SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving
• A review of mobile robot motion planning methods: from classical motion planning workflows to reinforcement learning-based architectures
• Model-based Reinforcement Learning for Decentralized Multiagent Rendezvous
• Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Survey
• Hierarchical RNNs-Based Transformers MADDPG for Mixed Cooperative-Competitive Environments
• Soft Hierarchical Graph Recurrent Networks for Many-Agent Partially Observable Environments
• Minimizing Communication while Maximizing Performance in Multi-Agent Reinforcement Learning
• On Memory Mechanism in Multi-Agent Reinforcement Learning