Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents

Abstract

The emergence of complex life on Earth is often attributed to the arms race that ensued from a huge number of organisms all competing for finite resources. We present an artificial intelligence research environment, inspired by the human game genre of MMORPGs (Massively Multiplayer Online Role-Playing Games, a.k.a. MMOs), that aims to simulate this setting in microcosm. As with MMORPGs and the real world alike, our environment is persistent and supports a large and variable number of agents. Our environment is well suited to the study of large-scale multiagent interaction: it requires that agents learn robust combat and navigation policies in the presence of large populations attempting to do the same. Baseline experiments reveal that population size magnifies and incentivizes the development of skillful behaviors and results in agents that outcompete agents trained in smaller populations. We further show that the policies of agents with unshared weights naturally diverge to fill different niches in order to avoid competition.

References in corpus (7)

• Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
• Mean Field Multi-Agent Reinforcement Learning
• Emergent Complexity via Multi-Agent Competition
• Gotta Learn Fast: A New Benchmark for Generalization in RL
• MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligence
• A Study of AI Population Dynamics with Million-agent Reinforcement Learning
• Playing Atari with Six Neurons

Cited by in corpus (17)

• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration
• RLCard: A Toolkit for Reinforcement Learning in Card Games
• Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning
• Action Semantics Network: Considering the Effects of Actions in Multiagent Systems
• Balancing of competitive two-player Game Levels with Reinforcement Learning
• Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence
• Collective Intelligence for Deep Learning: A Survey of Recent Developments
• Grounding Artificial Intelligence in the Origins of Human Behavior
• OpenSkill: A faster asymmetric multi-team, multiplayer rating system
• It might be balanced, but is it actually good? An Empirical Evaluation of Game Level Balancing
• TanksWorld: A Multi-Agent Environment for AI Safety Research
• Adaptive Synthetic Characters for Military Training
• A Conceptual Bio-Inspired Framework for the Evolution of Artificial General Intelligence
• Mimicking Evolution with Reinforcement Learning
• Neural MMO v1.3: A Massively Multiagent Game Environment for Training and Evaluating Neural Networks
• Learning Meta Representations for Agents in Multi-Agent Reinforcement Learning