Collaborative and Federated Black-box Optimization: A Bayesian Optimization Perspective

Abstract

We focus on collaborative and federated black-box optimization (BBOpt), where agents optimize their heterogeneous black-box functions through collaborative sequential experimentation. From a Bayesian optimization perspective, we address the fundamental challenges of distributed experimentation, heterogeneity, and privacy within BBOpt, and propose three unifying frameworks to tackle these issues: (i) a global framework where experiments are centrally coordinated, (ii) a local framework that allows agents to make decisions based on minimal shared information, and (iii) a predictive framework that enhances local surrogates through collaboration to improve decision-making. We categorize existing methods within these frameworks and highlight key open questions to unlock the full potential of federated BBOpt. Our overarching goal is to shift federated learning from its predominantly descriptive/predictive paradigm to a prescriptive one, particularly in the context of BBOpt - an inherently sequential decision-making problem.

References in corpus (11)

• Personalized Federated Learning: A Meta-Learning Approach
• Fair Resource Allocation in Federated Learning
• Batch Bayesian Optimization via Local Penalization
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• Federated Bayesian Optimization via Thompson Sampling
• GIFAIR-FL: A Framework for Group and Individual Fairness in Federated Learning
• Fed-ensemble: Improving Generalization through Model Ensembling in Federated Learning
• Federated Data Analytics: A Study on Linear Models
• Collaborative and Distributed Bayesian Optimization via Consensus: Showcasing the Power of Collaboration for Optimal Design
• Heterogeneous Matrix Factorization: When Features Differ by Datasets
• Personalized Dictionary Learning for Heterogeneous Datasets