Asynchronous Batch Bayesian Optimisation with Improved Local Penalisation

Abstract

Batch Bayesian optimisation (BO) has been successfully applied to hyperparameter tuning using parallel computing, but it is wasteful of resources: workers that complete jobs ahead of others are left idle. We address this problem by developing an approach, Penalising Locally for Asynchronous Bayesian Optimisation on $k$ workers (PLAyBOOK), for asynchronous parallel BO. We demonstrate empirically the efficacy of PLAyBOOK and its variants on synthetic tasks and a real-world problem. We undertake a comparison between synchronous and asynchronous BO, and show that asynchronous BO often outperforms synchronous batch BO in both wall-clock time and number of function evaluations.

Camera-ready version after incorporating reviewers' suggestions

Cited by in corpus (8)

• Model-based Asynchronous Hyperparameter and Neural Architecture Search
• Interpretable Neural Architecture Search via Bayesian Optimisation with Weisfeiler-Lehman Kernels
• Neural Architecture Generator Optimization
• Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits
• GIBBON: General-purpose Information-Based Bayesian OptimisatioN
• AgEBO-Tabular: Joint Neural Architecture and Hyperparameter Search with Autotuned Data-Parallel Training for Tabular Data
• A portfolio approach to massively parallel Bayesian optimization
• Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRL