The Hanabi Challenge: A New Frontier for AI Research

Abstract

From the early days of computing, games have been important testbeds for studying how well machines can do sophisticated decision making. In recent years, machine learning has made dramatic advances with artificial agents reaching superhuman performance in challenge domains like Go, Atari, and some variants of poker. As with their predecessors of chess, checkers, and backgammon, these game domains have driven research by providing sophisticated yet well-defined challenges for artificial intelligence practitioners. We continue this tradition by proposing the game of Hanabi as a new challenge domain with novel problems that arise from its combination of purely cooperative gameplay with two to five players and imperfect information. In particular, we argue that Hanabi elevates reasoning about the beliefs and intentions of other agents to the foreground. We believe developing novel techniques for such theory of mind reasoning will not only be crucial for success in Hanabi, but also in broader collaborative efforts, especially those with human partners. To facilitate future research, we introduce the open-source Hanabi Learning Environment, propose an experimental framework for the research community to evaluate algorithmic advances, and assess the performance of current state-of-the-art techniques.

arXiv comment32 pages, 5 figures, In Press (Artificial Intelligence)

References in corpus (16)

• The Arcade Learning Environment: An Evaluation Platform for General Agents
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker
• Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems
• Learning Multiagent Communication with Backpropagation
• Dopamine: A Research Framework for Deep Reinforcement Learning
• Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning
• Predicting Human Behavior in Unrepeated, Simultaneous-Move Games
• Learning Policy Representations in Multiagent Systems
• Emergent Communication through Negotiation
• DeepMind Lab
• Re-determinizing Information Set Monte Carlo Tree Search in Hanabi

Cited by in corpus (47)

• A Survey and Critique of Multiagent Deep Reinforcement Learning
• QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
• Compute Trends Across Three Eras of Machine Learning
• PettingZoo: Gym for Multi-Agent Reinforcement Learning
• OpenSpiel: A Framework for Reinforcement Learning in Games
• Making AI Intelligible: Philosophical Foundations
• Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning
• Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration
• On the Critical Role of Conventions in Adaptive Human-AI Collaboration
• Mind the gap: Challenges of deep learning approaches to Theory of Mind
• Emergent Multi-Agent Communication in the Deep Learning Era
• "Other-Play" for Zero-Shot Coordination
• Mimetic Models: Ethical Implications of AI that Acts Like You
• Learning to Play No-Press Diplomacy with Best Response Policy Iteration
• Open Problems in Cooperative AI
• Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabi
• Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks
• Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems
• The Pace of Artificial Intelligence Innovations: Speed, Talent, and Trial-and-Error
• Student of Games: A unified learning algorithm for both perfect and imperfect information games
• Neural Replicator Dynamics
• Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems
• Re-determinizing Information Set Monte Carlo Tree Search in Hanabi
• DREAM: Deep Regret minimization with Advantage baselines and Model-free learning
• Emergent Language: A Survey and Taxonomy
• Joint Policy Search for Multi-agent Collaboration with Imperfect Information
• Continuous Coordination As a Realistic Scenario for Lifelong Learning
• Neural Recursive Belief States in Multi-Agent Reinforcement Learning
• Adversarially Guided Self-Play for Adopting Social Conventions
• Generating and Adapting to Diverse Ad-Hoc Cooperation Agents in Hanabi
• Multiplayer Support for the Arcade Learning Environment
• Regret Bounds for Decentralized Learning in Cooperative Multi-Agent Dynamical Systems
• Theory of Mind for Deep Reinforcement Learning in Hanabi
• Evaluating the Rainbow DQN Agent in Hanabi with Unseen Partners
• Learning in Quantum Common-Interest Games and the Separability Problem
• Combining Theory of Mind and Abduction for Cooperation under Imperfect Information
• Centralized Model and Exploration Policy for Multi-Agent RL
• Off-Belief Learning
• Learned Belief Search: Efficiently Improving Policies in Partially Observable Settings
• Reinforcement Learning on Human Decision Models for Uniquely Collaborative AI Teammates
• Cooperative Bayesian Optimization for Imperfect Agents
• Creative Captioning: An AI Grand Challenge Based on the Dixit Board Game
• Interactive AI with a Theory of Mind
• Augmenting the action space with conventions to improve multi-agent cooperation in Hanabi
• Two Approaches to Building Collaborative, Task-Oriented Dialog Agents through Self-Play
• Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessian
• Modeling Sensorimotor Coordination as Multi-Agent Reinforcement Learning with Differentiable Communication