Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges

Abstract

Multi-agent reinforcement learning (MARL) is a widely used Artificial Intelligence (AI) technique. However, current studies and applications need to address its scalability, non-stationarity, and trustworthiness. This paper aims to review methods and applications and point out research trends and visionary prospects for the next decade. First, this paper summarizes the basic methods and application scenarios of MARL. Second, this paper outlines the corresponding research methods and their limitations on safety, robustness, generalization, and ethical constraints that need to be addressed in the practical applications of MARL. In particular, we believe that trustworthy MARL will become a hot research topic in the next decade. In addition, we suggest that considering human interaction is essential for the practical application of MARL in various societies. Therefore, this paper also analyzes the challenges while MARL is applied to human-machine interaction.

arXiv comment43 pages, 5 figures

References in corpus (63)

• Explaining and Harnessing Adversarial Examples
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Training language models to follow instructions with human feedback
• Trust Region Policy Optimization
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives
• Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
• Mean Field Multi-Agent Reinforcement Learning
• Actor-Attention-Critic for Multi-Agent Reinforcement Learning
• Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
• Scientific multi-agent reinforcement learning for wall-models of turbulent flows
• Explainable Deep Reinforcement Learning: State of the Art and Challenges
• Survey on reinforcement learning for language processing
• QPLEX: Duplex Dueling Multi-Agent Q-Learning
• Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic
• FACMAC: Factored Multi-Agent Centralised Policy Gradients
• A Review of Safe Reinforcement Learning: Methods, Theory and Applications
• CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning
• Cooperative and Distributed Reinforcement Learning of Drones for Field Coverage
• Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learning
• Stop-and-Go: Exploring Backdoor Attacks on Deep Reinforcement Learning-based Traffic Congestion Control Systems
• A Cooperative Multi-Agent Reinforcement Learning Framework for Resource Balancing in Complex Logistics Network
• Learning Efficient Multi-agent Communication: An Information Bottleneck Approach
• Invariant Causal Prediction for Block MDPs
• RoMFAC: A robust mean-field actor-critic reinforcement learning against adversarial perturbations on states
• Multi-Agent Reinforcement Learning in a Realistic Limit Order Book Market Simulation
• A Survey of Multi-Agent Deep Reinforcement Learning with Communication
• MSPM: A Modularized and Scalable Multi-Agent Reinforcement Learning-based System for Financial Portfolio Management
• Safe Multi-Agent Reinforcement Learning via Shielding
• Boundary-aware Supervoxel-level Iteratively Refined Interactive 3D Image Segmentation with Multi-agent Reinforcement Learning
• Multi-Agent Reinforcement Learning: A Report on Challenges and Approaches
• Online Deep Reinforcement Learning for Autonomous UAV Navigation and Exploration of Outdoor Environments
• Partially Observable Mean Field Multi-Agent Reinforcement Learning Based on Graph-Attention
• Enhancing the Robustness of QMIX against State-adversarial Attacks
• Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability
• Multi-Agent Constrained Policy Optimisation
• Fairness in Reinforcement Learning
• Robustness Testing for Multi-Agent Reinforcement Learning: State Perturbations on Critical Agents
• Learning Fair Policies in Decentralized Cooperative Multi-Agent Reinforcement Learning
• Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospects
• What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?
• Safe Deep Reinforcement Learning for Multi-Agent Systems with Continuous Action Spaces
• Interpretable AI for policy-making in pandemics
• Scalable Multi-Agent Reinforcement Learning for Warehouse Logistics with Robotic and Human Co-Workers
• Deceptive Reinforcement Learning for Privacy-Preserving Planning
• Sparse Adversarial Attack in Multi-agent Reinforcement Learning
• Execution-based Code Generation using Deep Reinforcement Learning
• A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learning
• BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning
• Asynchronous Distributed Bilevel Optimization
• Distributed Distributionally Robust Optimization with Non-Convex Objectives
• Event-Triggered Multi-agent Reinforcement Learning with Communication under Limited-bandwidth Constraint
• Attacking c-MARL More Effectively: A Data Driven Approach
• PolicyCleanse: Backdoor Detection and Mitigation in Reinforcement Learning
• Model based Multi-agent Reinforcement Learning with Tensor Decompositions
• Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influence
• MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Trees
• PP-MARL: Efficient Privacy-Preserving Multi-Agent Reinforcement Learning for Cooperative Intelligence in Communications
• Reinforcement Learning with Stepwise Fairness Constraints
• Concept Learning for Interpretable Multi-Agent Reinforcement Learning