Emergent Complexity via Multi-Agent Competition
arXiv:1710.03748
Abstract
Reinforcement learning algorithms can train agents that solve problems in complex, interesting environments. Normally, the complexity of the trained agent is closely related to the complexity of the environment. This suggests that a highly capable agent requires a complex environment for training. In this paper, we point out that a competitive multi-agent environment trained with self-play can produce behaviors that are far more complex than the environment itself. We also point out that such environments come with a natural curriculum, because for any skill level, an environment full of agents of this level will have the right level of difficulty. This work introduces several competitive multi-agent environments where agents compete in a 3D world with simulated physics. The trained agents learn a wide variety of complex and interesting skills, even though the environment themselves are relatively simple. The skills include behaviors such as running, blocking, ducking, tackling, fooling opponents, kicking, and defending using both arms and legs. A highlight of the learned behaviors can be found here: https://goo.gl/eR7fbX
Published as a conference paper at ICLR 2018
References in corpus (3)
Cited by in corpus (55)
- Dota 2 with Large Scale Deep Reinforcement Learning
- A Survey and Critique of Multiagent Deep Reinforcement Learning
- Unity: A General Platform for Intelligent Agents
- A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
- Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning
- Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments
- A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning
- Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions
- Universal Planning Networks
- AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
- Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents
- Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions
- Relative Entropy Regularized Policy Iteration
- Ranked Reward: Enabling Self-Play Reinforcement Learning for Combinatorial Optimization
- Automatic Curriculum Learning through Value Disagreement
- Neural Graph Evolution: Towards Efficient Automatic Robot Design
- Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input
- AI Research Considerations for Human Existential Safety (ARCHES)
- Mastering Complex Control in MOBA Games with Deep Reinforcement Learning
- Generalization in Transfer Learning
- Learning to Play No-Press Diplomacy with Best Response Policy Iteration
- Collaborative Visual Navigation
- Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabi
- Scratch that! An Evolution-based Adversarial Attack against Neural Networks
- Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarios
- SMiRL: Surprise Minimizing Reinforcement Learning in Unstable Environments
- Algorithms in Multi-Agent Systems: A Holistic Perspective from Reinforcement Learning and Game Theory
- TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning
- Neural Replicator Dynamics
- Multi-Agent Deep Reinforcement Learning for Liquidation Strategy Analysis
- Continual Match Based Training in Pommerman: Technical Report
- Robust Opponent Modeling via Adversarial Ensemble Reinforcement Learning in Asymmetric Imperfect-Information Games
- BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning
- Application of Self-Play Reinforcement Learning to a Four-Player Game of Imperfect Information
- Collective Intelligence for Deep Learning: A Survey of Recent Developments
- Learning Zero-Sum Simultaneous-Move Markov Games Using Function Approximation and Correlated Equilibrium
- XDO: A Double Oracle Algorithm for Extensive-Form Games
- Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks
- FormulaZero: Distributionally Robust Online Adaptation via Offline Population Synthesis
- Robustness to Adversarial Attacks in Learning-Enabled Controllers
- Arena: a toolkit for Multi-Agent Reinforcement Learning
- Multi-Agent Deep Reinforcement Learning with Adaptive Policies
- PAC Guarantees for Cooperative Multi-Agent Reinforcement Learning with Restricted Communication
- Competing AI: How does competition feedback affect machine learning?
- Distributed Deep Reinforcement Learning: An Overview
- Mimicking Evolution with Reinforcement Learning
- Continual Reinforcement Learning with Diversity Exploration and Adversarial Self-Correction
- Reinforcement Learning Agents for Ubisoft's Roller Champions
- Reinforcement Learning In Two Player Zero Sum Simultaneous Action Games
- Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks
- Interaction-Aware Multi-Agent Reinforcement Learning for Mobile Agents with Individual Goals
- Autonomous Industrial Management via Reinforcement Learning: Self-Learning Agents for Decision-Making -- A Review
- Behaviour-conditioned policies for cooperative reinforcement learning tasks
- On the potential for open-endedness in neural networks
- Deep RL Agent for a Real-Time Action Strategy Game