Bayesian Nonparametric Federated Learning of Neural Networks

Abstract

In federated learning problems, data is scattered across different servers and exchanging or pooling it is often impractical or prohibited. We develop a Bayesian nonparametric framework for federated learning with neural networks. Each data server is assumed to provide local neural network weights, which are modeled through our framework. We then develop an inference approach that allows us to synthesize a more expressive global network without additional supervision, data pooling and with as few as a single communication round. We then demonstrate the efficacy of our approach on federated learning problems simulated from two popular image classification datasets.

arXiv commentICML 2019

Cited by in corpus (45)

• Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
• From Distributed Machine Learning to Federated Learning: A Survey
• SecureBoost: A Lossless Federated Learning Framework
• A Field Guide to Federated Optimization
• Decentralized Federated Learning through Proxy Model Sharing
• A Federated Learning Aggregation Algorithm for Pervasive Computing: Evaluation and Comparison
• Variational Federated Multi-Task Learning
• Federated Learning Based on Dynamic Regularization
• IBM Federated Learning: an Enterprise Framework White Paper V0.1
• Federated Learning with Matched Averaging
• Federated Learning with Spiking Neural Networks
• Federated Learning on Non-IID Data Silos: An Experimental Study
• FedMix: Approximation of Mixup under Mean Augmented Federated Learning
• FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout
• A first look into the carbon footprint of federated learning
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data
• Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data
• Federated Continual Learning with Weighted Inter-client Transfer
• Federated Learning Meets Multi-objective Optimization
• Practical Federated Gradient Boosting Decision Trees
• SemiFed: Semi-supervised Federated Learning with Consistency and Pseudo-Labeling
• A Systematic Literature Review on Federated Learning: From A Model Quality Perspective
• Model-Contrastive Federated Learning
• Adversarial training in communication constrained federated learning
• Gradient Sparsification for Efficient Wireless Federated Learning with Differential Privacy
• Device Heterogeneity in Federated Learning: A Superquantile Approach
• Specialized federated learning using a mixture of experts
• Federated Learning with Superquantile Aggregation for Heterogeneous Data
• Personalized Federated Learning with Gaussian Processes
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning
• Hybrid Federated Learning: Algorithms and Implementation
• Federated Learning -- Methods, Applications and beyond
• RelaySum for Decentralized Deep Learning on Heterogeneous Data
• Distributed Computation for Marginal Likelihood based Model Choice
• FedHealth 2: Weighted Federated Transfer Learning via Batch Normalization for Personalized Healthcare
• FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis
• Federated CycleGAN for Privacy-Preserving Image-to-Image Translation
• A Differentially Private Probabilistic Framework for Modeling the Variability Across Federated Datasets of Heterogeneous Multi-View Observations
• Distribution-Free Federated Learning with Conformal Predictions
• An Operator Splitting View of Federated Learning
• Scalable inference of topic evolution via models for latent geometric structures
• Probabilistic Federated Learning of Neural Networks Incorporated with Global Posterior Information
• An Expectation-Maximization Perspective on Federated Learning
• Fed2: Feature-Aligned Federated Learning
• FLBench: A Benchmark Suite for Federated Learning