Experimenting, Fast and Slow: Bayesian Optimization of Long-term Outcomes with Online Experiments

Abstract

Online experiments in internet systems, also known as A/B tests, are used for a wide range of system tuning problems, such as optimizing recommender system ranking policies and learning adaptive streaming controllers. Decision-makers generally wish to optimize for long-term treatment effects of the system changes, which often requires running experiments for a long time as short-term measurements can be misleading due to non-stationarity in treatment effects over time. The sequential experimentation strategies--which typically involve several iterations--can be prohibitively long in such cases. We describe a novel approach that combines fast experiments (e.g., biased experiments run only for a few hours or days) and/or offline proxies (e.g., off-policy evaluation) with long-running, slow experiments to perform sequential, Bayesian optimization over large action spaces in a short amount of time.

References in corpus (4)

• Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020
• Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement
• Bayesian Optimization for Policy Search via Online-Offline Experimentation
• Joint Entropy Search for Maximally-Informed Bayesian Optimization