Scaling Scaling Laws with Board Games
arXiv:2104.03113
Abstract
The largest experiments in machine learning now require resources far beyond the budget of all but a few institutions. Fortunately, it has recently been shown that the results of these huge experiments can often be extrapolated from the results of a sequence of far smaller, cheaper experiments. In this work, we show that not only can the extrapolation be done based on the size of the model, but on the size of the problem as well. By conducting a sequence of experiments using AlphaZero and Hex, we show that the performance achievable with a fixed amount of compute degrades predictably as the game gets larger and harder. Along with our main result, we further show that the test-time and train-time compute available to an agent can be traded off while maintaining performance.
References in corpus (22)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Scaling Laws for Neural Language Models
- Dota 2 with Large Scale Deep Reinforcement Learning
- Deep Learning Scaling is Predictable, Empirically
- Thinking Fast and Slow with Deep Learning and Tree Search
- Scaling Laws for Autoregressive Generative Modeling
- An Empirical Model of Large-Batch Training
- OpenSpiel: A Framework for Reinforcement Learning in Games
- Accelerated Methods for Deep Reinforcement Learning
- A Constructive Prediction of the Generalization Error Across Scales
- Accelerating Self-Play Learning in Go
- Phasic Policy Gradient
- ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero
- Trivializations for Gradient-Based Optimization on Manifolds
- Re-evaluating Evaluation
- Monte-Carlo Tree Search as Regularized Policy Optimization
- Scaling Laws for Transfer
- Multiagent Evaluation under Incomplete Information
- On the Predictability of Pruning Across Scales
- Accelerating Reinforcement Learning through GPU Atari Emulation
- Real World Games Look Like Spinning Tops
- Learning Curve Theory