Active Federated Learning

Abstract

Federated Learning allows for population level models to be trained without centralizing client data by transmitting the global model to clients, calculating gradients locally, then averaging the gradients. Downloading models and uploading gradients uses the client's bandwidth, so minimizing these transmission costs is important. The data on each client is highly variable, so the benefit of training on different clients may differ dramatically. To exploit this we propose Active Federated Learning, where in each round clients are selected not uniformly at random, but with a probability conditioned on the current model and the data on the client to maximize efficiency. We propose a cheap, simple and intuitive sampling scheme which reduces the number of required training iterations by 20-70% while maintaining the same model accuracy, and which mimics well known resampling techniques under certain conditions.

References in corpus (4)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• How To Backdoor Federated Learning
• Expanding the Reach of Federated Learning by Reducing Client Resource Requirements

Cited by in corpus (11)

• Towards Fairness-Aware Federated Learning
• FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systems
• Towards Utilizing Unlabeled Data in Federated Learning: A Survey and Prospective
• Adaptive Client Selection with Personalization for Communication Efficient Federated Learning
• Data-Quality Based Scheduling for Federated Edge Learning
• Sustainable Federated Learning
• Wireless for Machine Learning
• Is Shapley Value fair? Improving Client Selection for Mavericks in Federated Learning
• On Large-Cohort Training for Federated Learning
• Dubhe: Towards Data Unbiasedness with Homomorphic Encryption in Federated Learning Client Selection
• FedProf: Selective Federated Learning with Representation Profiling