ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero

Abstract

The AlphaGo, AlphaGo Zero, and AlphaZero series of algorithms are remarkable demonstrations of deep reinforcement learning's capabilities, achieving superhuman performance in the complex game of Go with progressively increasing autonomy. However, many obstacles remain in the understanding of and usability of these promising approaches by the research community. Toward elucidating unresolved mysteries and facilitating future research, we propose ELF OpenGo, an open-source reimplementation of the AlphaZero algorithm. ELF OpenGo is the first open-source Go AI to convincingly demonstrate superhuman performance with a perfect (20:0) record against global top professionals. We apply ELF OpenGo to conduct extensive ablation studies, and to identify and analyze numerous interesting phenomena in both the model training and in the gameplay inference procedures. Our code, models, selfplay datasets, and auxiliary data are publicly available at https://ai.facebook.com/tools/elf-opengo/.

arXiv commentPublished as a conference paper at ICML 2019. This version contains supplementary appendices

References in corpus (5)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Move Evaluation in Go Using Deep Convolutional Neural Networks
• AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search
• Demystifying AlphaGo Zero as AlphaGo GAN
• From Gameplay to Symbolic Reasoning: Learning SAT Solver Heuristics in the Style of Alpha(Go) Zero

Cited by in corpus (12)

• The Fundamental Principles of Reproducibility
• DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames
• Sample-Efficient Neural Architecture Search by Learning Action Space
• The Scientific Method in the Science of Machine Learning
• Rethinking "Batch" in BatchNorm
• Multiplayer AlphaZero
• Neural Architecture Search using Deep Neural Networks and Monte Carlo Tree Search
• Scaling Scaling Laws with Board Games
• Joint Policy Search for Multi-agent Collaboration with Imperfect Information
• Deep Synoptic Monte Carlo Planning in Reconnaissance Blind Chess
• Mobile Networks for Computer Go
• Multiple Policy Value Monte Carlo Tree Search