FedPrune: Towards Inclusive Federated Learning

Abstract

Federated learning (FL) is a distributed learning technique that trains a shared model over distributed data in a privacy-preserving manner. Unfortunately, FL's performance degrades when there is (i) variability in client characteristics in terms of computational and memory resources (system heterogeneity) and (ii) non-IID data distribution across clients (statistical heterogeneity). For example, slow clients get dropped in FL schemes, such as Federated Averaging (FedAvg), which not only limits overall learning but also biases results towards fast clients. We propose FedPrune; a system that tackles this challenge by pruning the global model for slow clients based on their device characteristics. By doing so, slow clients can train a small model quickly and participate in FL which increases test accuracy as well as fairness. By using insights from Central Limit Theorem, FedPrune incorporates a new aggregation technique that achieves robust performance over non-IID data. Experimental evaluation shows that Fed- Prune provides robust convergence and better fairness compared to Federated Averaging.

References in corpus (10)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Towards Federated Learning at Scale: System Design
• Federated Optimization in Heterogeneous Networks
• LEAF: A Benchmark for Federated Settings
• Ditto: Fair and Robust Federated Learning Through Personalization
• Fairness Without Demographics in Repeated Loss Minimization
• HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients
• A Field Guide to Federated Optimization
• Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
• Federated Learning Meets Multi-objective Optimization