Distributionally Robust Bayesian Optimization
arXiv:2002.09038
Abstract
Robustness to distributional shift is one of the key challenges of contemporary machine learning. Attaining such robustness is the goal of distributionally robust optimization, which seeks a solution to an optimization problem that is worst-case robust under a specified distributional shift of an uncontrolled covariate. In this paper, we study such a problem when the distributional shift is measured via the maximum mean discrepancy (MMD). For the setting of zeroth-order, noisy optimization, we present a novel distributionally robust Bayesian optimization algorithm (DRBO). Our algorithm provably obtains sub-linear robust regret in various settings that differ in how the uncertain covariate is observed. We demonstrate the robust performance of our method on both synthetic and real-world benchmarks.
arXiv commentAccepted at AISTATS 2020
References in corpus (14)
- Certifying Some Distributional Robustness with Principled Adversarial Training
- Bayesian Optimization with Unknown Constraints
- Max-value Entropy Search for Efficient Bayesian Optimization
- Demystifying MMD GANs
- Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy
- Finite-Time Analysis of Kernelised Contextual Bandits
- A Kernel Test of Goodness of Fit
- Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspaces
- Adversarially Robust Optimization with Gaussian Processes
- Time-Varying Gaussian Process Bandit Optimization
- Truncated Variance Reduction: A Unified Approach to Bayesian Optimization and Level-Set Estimation
- Practical Bayesian optimization in the presence of outliers
- Distributionally Robust Submodular Maximization
- Distributionally Robust Bayesian Quadrature Optimization
Cited by in corpus (7)
- Adaptive Sampling for Stochastic Risk-Averse Learning
- Kernel Distributionally Robust Optimization
- On Lower Bounds for Standard and Robust Gaussian Process Bandit Optimization
- Expert-Calibrated Learning for Online Optimization with Switching Costs
- Value-at-Risk Optimization with Gaussian Processes
- Unsupervised Learning of Debiased Representations with Pseudo-Attributes
- Neural Contextual Bandits without Regret