Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling

Abstract

In this paper, we propose \texttt{FGPR}: a Federated Gaussian process ($\mathcal{G}\mathcal{P}$) regression framework that uses an averaging strategy for model aggregation and stochastic gradient descent for local client computations. Notably, the resulting global model excels in personalization as \texttt{FGPR} jointly learns a global $\mathcal{G}\mathcal{P}$ prior across all clients. The predictive posterior then is obtained by exploiting this prior and conditioning on local data which encodes personalized features from a specific client. Theoretically, we show that \texttt{FGPR} converges to a critical point of the full log-likelihood function, subject to statistical error. Through extensive case studies we show that \texttt{FGPR} excels in a wide range of applications and is a promising approach for privacy-preserving multi-fidelity data modeling.

arXiv comment56 pages. Under Review

References in corpus (11)

• Overcoming catastrophic forgetting in neural networks
• Federated Learning: Challenges, Methods, and Future Directions
• Gaussian Processes for Data-Efficient Learning in Robotics and Control
• FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data
• Quantile Regression Under Memory Constraint
• Review of multi-fidelity models
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• First-order Newton-type Estimator for Distributed Estimation and Inference
• GIFAIR-FL: A Framework for Group and Individual Fairness in Federated Learning
• Robust Experimental Designs for Model Calibration
• Modular Gaussian Processes for Transfer Learning