Federated Learning from Small Datasets

Abstract

Federated learning allows multiple parties to collaboratively train a joint model without sharing local data. This enables applications of machine learning in settings of inherently distributed, undisclosable data such as in the medical domain. In practice, joint training is usually achieved by aggregating local models, for which local training objectives have to be in expectation similar to the joint (global) objective. Often, however, local datasets are so small that local objectives differ greatly from the global objective, resulting in federated learning to fail. We propose a novel approach that intertwines model aggregations with permutations of local models. The permutations expose each local model to a daisy chain of local datasets resulting in more efficient training in data-sparse domains. This enables training on extremely small local datasets, such as patient data across hospitals, while retaining the training efficiency and privacy benefits of federated learning.

References in corpus (10)

• Scikit-learn: Machine Learning in Python
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Searching for Exotic Particles in High-Energy Physics with Deep Learning
• Robust Aggregation for Federated Learning
• Differentially Private Federated Learning: A Client Level Perspective
• Can You Really Backdoor Federated Learning?
• FedBN: Federated Learning on Non-IID Features via Local Batch Normalization
• FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
• On the Convergence of Local Descent Methods in Federated Learning
• Federated Learning with Matched Averaging