Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning

Abstract

Recent developments in deep reinforcement learning are concerned with creating decision-making agents which can perform well in various complex domains. A particular approach which has received increasing attention is multi-agent reinforcement learning, in which multiple agents learn concurrently to coordinate their actions. In such multi-agent environments, additional learning problems arise due to the continually changing decision-making policies of agents. This paper surveys recent works that address the non-stationarity problem in multi-agent deep reinforcement learning. The surveyed methods range from modifications in the training procedure, such as centralized training, to learning representations of the opponent's policy, meta-learning, communication, and decentralized learning. The survey concludes with a list of open problems and possible lines of future research.

References in corpus (1)

• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

Cited by in corpus (9)

• Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning
• Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systems
• Agent-Based Modelling for Urban Analytics: State of the Art and Challenges
• Delay-Aware Multi-Agent Reinforcement Learning for Cooperative and Competitive Environments
• Fever Basketball: A Complex, Flexible, and Asynchronized Sports Game Environment for Multi-agent Reinforcement Learning
• Deception in Social Learning: A Multi-Agent Reinforcement Learning Perspective
• Variational Autoencoders for Opponent Modeling in Multi-Agent Systems
• Multi-Agent Deep Reinforcement Learning with Adaptive Policies
• Coordinated Proximal Policy Optimization