Bayesian Optimization in AlphaGo

Abstract

During the development of AlphaGo, its many hyper-parameters were tuned with Bayesian optimization multiple times. This automatic tuning process resulted in substantial improvements in playing strength. For example, prior to the match with Lee Sedol, we tuned the latest AlphaGo agent and this improved its win-rate from 50% to 66.5% in self-play games. This tuned version was deployed in the final match. Of course, since we tuned AlphaGo many times during its development cycle, the compounded contribution was even higher than this percentage. It is our hope that this brief case study will be of interest to Go fans, and also provide Bayesian optimization practitioners with some insights and inspiration.

References in corpus (1)

• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning

Cited by in corpus (28)

• Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
• Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML
• c-TPE: Tree-structured Parzen Estimator with Inequality Constraints for Expensive Hyperparameter Optimization
• Neural Architecture Generator Optimization
• Optimization of the Shape of a Hydrokinetic Turbine's Draft Tube and Hub Assembly Using Design-by-Morphing with Bayesian Optimization
• Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits
• Using a thousand optimization tasks to learn hyperparameter search strategies
• Fast Efficient Hyperparameter Tuning for Policy Gradients
• PED-ANOVA: Efficiently Quantifying Hyperparameter Importance in Arbitrary Subspaces
• Knowing The What But Not The Where in Bayesian Optimization
• Speeding Up Multi-Objective Hyperparameter Optimization by Task Similarity-Based Meta-Learning for the Tree-Structured Parzen Estimator
• GIBBON: General-purpose Information-Based Bayesian OptimisatioN
• Decoding Non-Linearity and Complexity: Deep Tabular Learning Approaches for Materials Science
• Meta-Surrogate Benchmarking for Hyperparameter Optimization
• Achieving Real-Time LiDAR 3D Object Detection on a Mobile Device
• Neural Process for Black-Box Model Optimization Under Bayesian Framework
• NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration
• Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search
• Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRL
• Probabilistic Rollouts for Learning Curve Extrapolation Across Hyperparameter Settings
• Bayesian Optimization for Iterative Learning
• Optimality Criteria for Probabilistic Numerical Methods
• Hyperparameter Transfer Across Developer Adjustments
• Mixed Variable Bayesian Optimization with Frequency Modulated Kernels
• Active Offline Policy Selection
• Achieving Real-Time Object Detection on MobileDevices with Neural Pruning Search
• Bayesian optimization for backpropagation in Monte-Carlo tree search
• Multifidelity Bayesian Optimization for Binomial Output