Oort: Efficient Federated Learning via Guided Participant Selection

Abstract

Federated Learning (FL) is an emerging direction in distributed machine learning (ML) that enables in-situ model training and testing on edge data. Despite having the same end goals as traditional ML, FL executions differ significantly in scale, spanning thousands to millions of participating devices. As a result, data characteristics and device capabilities vary widely across clients. Yet, existing efforts randomly select FL participants, which leads to poor model and system efficiency. In this paper, we propose Oort to improve the performance of federated training and testing with guided participant selection. With an aim to improve time-to-accuracy performance in model training, Oort prioritizes the use of those clients who have both data that offers the greatest utility in improving model accuracy and the capability to run training quickly. To enable FL developers to interpret their results in model testing, Oort enforces their requirements on the distribution of participant data while improving the duration of federated testing by cherry-picking clients. Our evaluation shows that, compared to existing participant selection mechanisms, Oort improves time-to-accuracy performance by 1.2x-14.1x and final model accuracy by 1.3%-9.8%, while efficiently enforcing developer-specified model testing criteria at the scale of millions of clients.

References in corpus (8)

• Agnostic Federated Learning
• Prio: Private, Robust, and Scalable Computation of Aggregate Statistics
• Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies
• Biased Importance Sampling for Deep Neural Network Training
• Federated Learning with Only Positive Labels
• Federated Learning for Ranking Browser History Suggestions
• Federated Learning of N-gram Language Models
• Training Production Language Models without Memorizing User Data

Cited by in corpus (6)

• Federated Few-Shot Learning for Mobile NLP
• Ed-Fed: A generic federated learning framework with resource-aware client selection for edge devices
• Accelerating MoE Model Inference with Expert Sharding
• Practical Federated Learning without a Server
• Canoe : A System for Collaborative Learning for Neural Nets
• FedOptima: Optimizing Resource Utilization in Federated Learning