Differentially Private Bayesian Optimization

Abstract

Bayesian optimization is a powerful tool for fine-tuning the hyper-parameters of a wide variety of machine learning models. The success of machine learning has led practitioners in diverse real-world settings to learn classifiers for practical problems. As machine learning becomes commonplace, Bayesian optimization becomes an attractive method for practitioners to automate the process of classifier hyper-parameter tuning. A key observation is that the data used for tuning models in these settings is often sensitive. Certain data such as genetic predisposition, personal email statistics, and car accident history, if not properly private, may be at risk of being inferred from Bayesian optimization outputs. To address this, we introduce methods for releasing the best hyper-parameters and classifier accuracy privately. Leveraging the strong theoretical guarantees of differential privacy and known Bayesian optimization convergence bounds, we prove that under a GP assumption these private quantities are also near-optimal. Finally, even if this assumption is not satisfied, we can use different smoothness guarantees to protect privacy.

References in corpus (3)

• Practical Bayesian Optimization of Machine Learning Algorithms
• Hybrid Batch Bayesian Optimization
• Exponential Regret Bounds for Gaussian Process Bandits with Deterministic Observations

Cited by in corpus (8)

• Federated Bayesian Optimization via Thompson Sampling
• Learning Rate Adaptation for Federated and Differentially Private Learning
• Privacy-preserving Q-Learning with Functional Noise in Continuous State Spaces
• Accuracy, Interpretability, and Differential Privacy via Explainable Boosting
• Local Differential Privacy for Bayesian Optimization
• Privacy Enhancing Machine Learning via Removal of Unwanted Dependencies
• Technologies for Trustworthy Machine Learning: A Survey in a Socio-Technical Context
• No-Regret Algorithms for Private Gaussian Process Bandit Optimization