Accelerating Self-Play Learning in Go
arXiv:1902.10565
Abstract
By introducing several improvements to the AlphaZero process and architecture, we greatly accelerate self-play learning in Go, achieving a 50x reduction in computation over comparable methods. Like AlphaZero and replications such as ELF OpenGo and Leela Zero, our bot KataGo only learns from neural-net-guided Monte Carlo tree search self-play. But whereas AlphaZero required thousands of TPUs over several days and ELF required thousands of GPUs over two weeks, KataGo surpasses ELF's final model after only 19 days on fewer than 30 GPUs. Much of the speedup involves non-domain-specific improvements that might directly transfer to other problems. Further gains from domain-specific techniques reveal the remaining efficiency gap between the best methods and purely general methods such as AlphaZero. Our work is a step towards making learning in state spaces as large as Go possible without large-scale computational resources.
arXiv comment28 pages including appendices, 8 figures, 7 tables. Presented at AAAI20-RLG workshop. (this version: updated an email address)
References in corpus (4)
Cited by in corpus (11)
- Superhuman Artificial Intelligence Can Improve Human Decision Making by Increasing Novelty
- Score vs. Winrate in Score-Based Games: which Reward for Reinforcement Learning?
- Scaling Scaling Laws with Board Games
- SAI: a Sensible Artificial Intelligence that plays with handicap and targets high scores in 9x9 Go (extended version)
- Transfer of Fully Convolutional Policy-Value Networks Between Games and Game Variants
- Adaptive Warm-Start MCTS in AlphaZero-like Deep Reinforcement Learning
- Learning to Stop: Dynamic Simulation Monte-Carlo Tree Search
- AlphaViT: A flexible game-playing AI for multiple games and variable board sizes
- Mobile Networks for Computer Go
- Self-Play Learning Without a Reward Metric
- Deep Synoptic Monte Carlo Planning in Reconnaissance Blind Chess