Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications

Abstract

Reinforcement learning (RL) algorithms have been around for decades and employed to solve various sequential decision-making problems. These algorithms however have faced great challenges when dealing with high-dimensional environments. The recent development of deep learning has enabled RL methods to drive optimal policies for sophisticated and capable agents, which can perform efficiently in these challenging environments. This paper addresses an important aspect of deep RL related to situations that require multiple agents to communicate and cooperate to solve complex tasks. A survey of different approaches to problems related to multi-agent deep RL (MADRL) is presented, including non-stationarity, partial observability, continuous state and action spaces, multi-agent training schemes, multi-agent transfer learning. The merits and demerits of the reviewed methods will be analyzed and discussed, with their corresponding applications explored. It is envisaged that this review provides insights about various MADRL methods and can lead to future development of more robust and highly useful multi-agent learning methods for solving real-world problems.

References in corpus (42)

• Continuous control with deep reinforcement learning
• Overcoming catastrophic forgetting in neural networks
• A Brief Survey of Deep Reinforcement Learning
• Trust Region Policy Optimization
• Prioritized Experience Replay
• Dueling Network Architectures for Deep Reinforcement Learning
• End-to-End Training of Deep Visuomotor Policies
• Progressive Neural Networks
• The Arcade Learning Environment: An Evaluation Platform for General Agents
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Benchmarking Deep Reinforcement Learning for Continuous Control
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• Deep Recurrent Q-Learning for Partially Observable MDPs
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Deep Reinforcement Learning: An Overview
• Counterfactual Multi-Agent Policy Gradients
• Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
• Cooperative Inverse Reinforcement Learning
• Continuous Deep Q-Learning with Model-based Acceleration
• Multi-agent Reinforcement Learning in Sequential Social Dilemmas
• Memory-based control with recurrent neural networks
• A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
• Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability
• A Multi-Objective Deep Reinforcement Learning Framework
• Opponent Modeling in Deep Reinforcement Learning
• Maintaining cooperation in complex social dilemmas using deep reinforcement learning
• Playing FPS Games with Deep Reinforcement Learning
• Deep Attention Recurrent Q-Network
• Guided Deep Reinforcement Learning for Swarm Systems
• Policy Distillation
• Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks
• Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning
• On Improving Deep Reinforcement Learning for POMDPs
• A multi-agent reinforcement learning model of common-pool resource appropriation
• Efficient Model-Based Deep Reinforcement Learning with Variational State Tabulation
• Revisiting the Master-Slave Architecture in Multi-Agent Deep Reinforcement Learning
• A Deep Policy Inference Q-Network for Multi-Agent Systems
• Federated Control with Hierarchical Multi-Agent Deep Reinforcement Learning
• Multi-agent Deep Reinforcement Learning with Extremely Noisy Observations
• A Human Mixed Strategy Approach to Deep Reinforcement Learning
• Efficient Collaborative Multi-Agent Deep Reinforcement Learning for Large-Scale Fleet Management
• The Bottleneck Simulator: A Model-based Deep Reinforcement Learning Approach

Cited by in corpus (86)

• Artificial Intelligence in the Creative Industries: A Review
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Deep Reinforcement Learning for Cyber Security
• Exploration in Deep Reinforcement Learning: A Survey
• A Review of Deep Reinforcement Learning for Smart Building Energy Management
• Cooperative Multi-Agent Deep Reinforcement Learning for Reliable Surveillance via Autonomous Multi-UAV Control
• A Survey of Deep RL and IL for Autonomous Driving Policy Learning
• Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem
• A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
• Five Facets of 6G: Research Challenges and Opportunities
• Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence
• Generative chemistry: drug discovery with deep learning generative models
• Game-Theoretic Multiagent Reinforcement Learning
• Modeling Human Driving Behavior through Generative Adversarial Imitation Learning
• Systemic formalisation of Cyber-Physical-Social System (CPSS): A systematic literature review
• Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey
• Review: Deep Learning in Electron Microscopy
• Energy Management Based on Multi-Agent Deep Reinforcement Learning for A Multi-Energy Industrial Park
• Machine Learning for Condensed Matter Physics
• A Survey on Energy Optimization Techniques in UAV-Based Cellular Networks: From Conventional to Machine Learning Approaches
• Reward Design for Driver Repositioning Using Multi-Agent Reinforcement Learning
• Deep Reinforcement Learning for Radio Resource Allocation and Management in Next Generation Heterogeneous Wireless Networks: A Survey
• Multi-Agent Reinforcement Learning for Markov Routing Games: A New Modeling Paradigm For Dynamic Traffic Assignment
• A Review of Safe Reinforcement Learning Methods for Modern Power Systems
• A Reinforcement Learning-based Economic Model Predictive Control Framework for Autonomous Operation of Chemical Reactors
• Learning Scheduling Algorithms for Data Processing Clusters
• Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning
• Federated Deep Reinforcement Learning for RIS-Assisted Indoor Multi-Robot Communication Systems
• Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges
• DRL-based Latency-Aware Network Slicing in O-RAN with Time-Varying SLAs
• A Deep Multi-Agent Reinforcement Learning Approach to Autonomous Separation Assurance
• Cellular Network Capacity and Coverage Enhancement with MDT Data and Deep Reinforcement Learning
• Interactive inference: a multi-agent model of cooperative joint actions
• Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexity
• Target Search and Navigation in Heterogeneous Robot Systems with Deep Reinforcement Learning
• BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication)
• Learning Multi-Agent Coordination through Connectivity-driven Communication
• Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxation
• Graph-attention-based Casual Discovery with Trust Region-navigated Clipping Policy Optimization
• Multi-Agent Trust Region Policy Optimization
• Multi-Agent Deep Reinforcement Learning for Request Dispatching in Distributed-Controller Software-Defined Networking
• Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems
• Cooperative Control of Mobile Robots with Stackelberg Learning
• A Deep Reinforcement Learning Approach for Global Routing
• The Emergence of Wireless MAC Protocols with Multi-Agent Reinforcement Learning
• Strategic Maneuver and Disruption with Reinforcement Learning Approaches for Multi-Agent Coordination
• A Deep Reinforcement Learning Approach for Security-Aware Service Acquisition in IoT
• Recommender systems and reinforcement learning for human-building interaction and context-aware support: A text mining-driven review of scientific literature
• Adaptive Target Localization under Uncertainty using Multi-Agent Deep Reinforcement Learning with Knowledge Transfer
• Blockchain-assisted Demonstration Cloning for Multi-Agent Deep Reinforcement Learning
• Combining the Projective Consciousness Model and Virtual Humans to assess ToM capacity in Virtual Reality: a proof-of-concept
• Online Multi-Objective Model-Independent Adaptive Tracking Mechanism for Dynamical Systems
• A Visual Communication Map for Multi-Agent Deep Reinforcement Learning
• Driving Tasks Transfer in Deep Reinforcement Learning for Decision-making of Autonomous Vehicles
• Multi-Agent Reinforcement Learning for Unmanned Aerial Vehicle Coordination by Multi-Critic Policy Gradient Optimization
• Multi-agent Reinforcement Learning-based In-place Scaling Engine for Edge-cloud Systems
• Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific research
• Multi-user Resource Control with Deep Reinforcement Learning in IoT Edge Computing
• Automated and Autonomous Experiment in Electron and Scanning Probe Microscopy
• Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Survey
• Reinforcement Learning Based Handwritten Digit Recognition with Two-State Q-Learning
• Review, Analysis and Design of a Comprehensive Deep Reinforcement Learning Framework
• Whittle Index Based User Association in Dense Millimeter Wave Networks
• Competitive Multi-Agent Deep Reinforcement Learning with Counterfactual Thinking
• Competing AI: How does competition feedback affect machine learning?
• Deep Q-Network Based Multi-agent Reinforcement Learning with Binary Action Agents
• Deep Reinforcement Learning for Joint Spectrum and Power Allocation in Cellular Networks
• MARLIM: Multi-Agent Reinforcement Learning for Inventory Management
• Multi-task Reinforcement Learning in Reproducing Kernel Hilbert Spaces via Cross-learning
• Towards Closing the Sim-to-Real Gap in Collaborative Multi-Robot Deep Reinforcement Learning
• Reinforcement Learning for Industrial Control Network Cyber Security Orchestration
• Graph-Embedded Multi-Agent Learning for Smart Reconfigurable THz MIMO-NOMA Networks
• Hierarchical RNNs-Based Transformers MADDPG for Mixed Cooperative-Competitive Environments
• Stability Analysis of Deep Reinforcement Learning for Multi-Agent Inspection in a Terrestrial Testbed
• Neural-iLQR: A Learning-Aided Shooting Method for Trajectory Optimization
• Peer-to-Peer Energy Trading in Dairy Farms using Multi-Agent Reinforcement Learning
• Controlling Complex Systems
• Distributed Safe Learning using an Invariance-based Safety Framework
• Optimal Operation of a Hydrogen-based Building Multi-Energy System Based on Deep Reinforcement Learning
• Battlesnake Challenge: A Multi-agent Reinforcement Learning Playground with Human-in-the-loop
• Ubiquitous Distributed Deep Reinforcement Learning at the Edge: Analyzing Byzantine Agents in Discrete Action Spaces
• MDLdroid: a ChainSGD-reduce Approach to Mobile Deep Learning for Personal Mobile Sensing
• Distributed Policy Gradient with Variance Reduction in Multi-Agent Reinforcement Learning
• Two-stage training algorithm for AI robot soccer
• Voting-Based Multi-Agent Reinforcement Learning for Intelligent IoT
• A Computational Model of Representation Learning in the Brain Cortex, Integrating Unsupervised and Reinforcement Learning