DistFL: Distribution-aware Federated Learning for Mobile Scenarios
arXiv:2110.11619
Abstract
Federated learning (FL) has emerged as an effective solution to decentralized and privacy-preserving machine learning for mobile clients. While traditional FL has demonstrated its superiority, it ignores the non-iid (independently identically distributed) situation, which widely exists in mobile scenarios. Failing to handle non-iid situations could cause problems such as performance decreasing and possible attacks. Previous studies focus on the "symptoms" directly, as they try to improve the accuracy or detect possible attacks by adding extra steps to conventional FL models. However, previous techniques overlook the root causes for the "symptoms": blindly aggregating models with the non-iid distributions. In this paper, we try to fundamentally address the issue by decomposing the overall non-iid situation into several iid clusters and conducting aggregation in each cluster. Specifically, we propose \textbf{DistFL}, a novel framework to achieve automated and accurate \textbf{Dist}ribution-aware \textbf{F}ederated \textbf{L}earning in a cost-efficient way. DistFL achieves clustering via extracting and comparing the \textit{distribution knowledge} from the uploaded models. With this framework, we are able to generate multiple personalized models with distinctive distributions and assign them to the corresponding clients. Extensive experiments on mobile scenarios with popular model architectures have demonstrated the effectiveness of DistFL.
arXiv commentThis paper has been accepted by IMWUT2021(Ubicomp)
References in corpus (26)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Distilling the Knowledge in a Neural Network
- Deep Learning with Differential Privacy
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
- FitNets: Hints for Thin Deep Nets
- Federated Learning with Non-IID Data
- Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
- Rényi Divergence and Kullback-Leibler Divergence
- A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
- Federated Learning for Mobile Keyboard Prediction
- Differentially Private Federated Learning: A Client Level Perspective
- Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications
- Applied Federated Learning: Improving Google Keyboard Query Suggestions
- Mitigating Sybils in Federated Learning Poisoning
- FedML: A Research Library and Benchmark for Federated Machine Learning
- A generic framework for privacy preserving deep learning
- LEAF: A Benchmark for Federated Settings
- Threats to Federated Learning: A Survey
- Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
- Think Locally, Act Globally: Federated Learning with Local and Global Representations
- Abnormal Client Behavior Detection in Federated Learning
- Salvaging Federated Learning by Local Adaptation
- Attack-Resistant Federated Learning with Residual-based Reweighting
- Robust Learning from Untrusted Sources
- TransTailor: Pruning the Pre-trained Model for Improved Transfer Learning