A coevolutionary approach to deep multi-agent reinforcement learning
arXiv:2104.05610
Abstract
Traditionally, Deep Artificial Neural Networks (DNN's) are trained through gradient descent. Recent research shows that Deep Neuroevolution (DNE) is also capable of evolving multi-million-parameter DNN's, which proved to be particularly useful in the field of Reinforcement Learning (RL). This is mainly due to its excellent scalability and simplicity compared to the traditional MDP-based RL methods. So far, DNE has only been applied to complex single-agent problems. As evolutionary methods are a natural choice for multi-agent problems, the question arises whether DNE can also be applied in a complex multi-agent setting. In this paper, we describe and validate a new approach based on Coevolution. To validate our approach, we benchmark two Deep Coevolutionary Algorithms on a range of multi-agent Atari games and compare our results against the results of Ape-X DQN. Our results show that these Deep Coevolutionary algorithms (1) can be successfully trained to play various games, (2) outperform Ape-X DQN in some of them, and therefore (3) show that Coevolution can be a viable approach to solving complex multi-agent decision-making problems.
References in corpus (11)
- Playing Atari with Deep Reinforcement Learning
- Trust Region Policy Optimization
- Improved Techniques for Training GANs
- Dota 2 with Large Scale Deep Reinforcement Learning
- Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- Solving Rubik's Cube with a Robot Hand
- Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
- Virtual to Real Reinforcement Learning for Autonomous Driving
- PettingZoo: Gym for Multi-Agent Reinforcement Learning
- Multiplayer Support for the Arcade Learning Environment