Battlesnake Challenge: A Multi-agent Reinforcement Learning Playground with Human-in-the-loop

Abstract

We present the Battlesnake Challenge, a framework for multi-agent reinforcement learning with Human-In-the-Loop Learning (HILL). It is developed upon Battlesnake, a multiplayer extension of the traditional Snake game in which 2 or more snakes compete for the final survival. The Battlesnake Challenge consists of an offline module for model training and an online module for live competitions. We develop a simulated game environment for the offline multi-agent model training and identify a set of baseline heuristics that can be instilled to improve learning. Our framework is agent-agnostic and heuristics-agnostic such that researchers can design their own algorithms, train their models, and demonstrate in the online Battlesnake competition. We validate the framework and baseline heuristics with our preliminary experiments. Our results show that agents with the proposed HILL methods consistently outperform agents without HILL. Besides, heuristics of reward manipulation had the best performance in the online competition. We open source our framework at https://github.com/awslabs/sagemaker-battlesnake-ai.

References in corpus (14)

• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• Mean Field Multi-Agent Reinforcement Learning
• DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable Feedback
• Deep Reinforcement Learning from Policy-Dependent Human Feedback
• Mastering Complex Control in MOBA Games with Deep Reinforcement Learning
• Agent-Agnostic Human-in-the-Loop Reinforcement Learning
• Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments
• Skynet: A Top Deep RL Agent in the Inaugural Pommerman Team Competition
• FRESH: Interactive Reward Shaping in High-Dimensional State Spaces using Human Feedback
• Accelerating Training in Pommerman with Imitation and Reinforcement Learning
• Leveraging Human Guidance for Deep Reinforcement Learning Tasks