Federated Learning on Non-IID Data: A Survey

Abstract

Federated learning is an emerging distributed machine learning framework for privacy preservation. However, models trained in federated learning usually have worse performance than those trained in the standard centralized learning mode, especially when the training data are not independent and identically distributed (Non-IID) on the local devices. In this survey, we pro-vide a detailed analysis of the influence of Non-IID data on both parametric and non-parametric machine learning models in both horizontal and vertical federated learning. In addition, cur-rent research work on handling challenges of Non-IID data in federated learning are reviewed, and both advantages and disadvantages of these approaches are discussed. Finally, we suggest several future research directions before concluding the paper.

References in corpus (25)

• Distilling the Knowledge in a Neural Network
• Towards Federated Learning at Scale: System Design
• Federated Learning with Personalization Layers
• FedMD: Heterogenous Federated Learning via Model Distillation
• EMNIST: an extension of MNIST to handwritten letters
• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• iDLG: Improved Deep Leakage from Gradients
• HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning
• Threats to Federated Learning: A Survey
• On the Convergence of Local Descent Methods in Federated Learning
• Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
• Overcoming Forgetting in Federated Learning on Non-IID Data
• Federated Learning with Matched Averaging
• A Framework for Evaluating Gradient Leakage Attacks in Federated Learning
• Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator
• Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer
• Multi-Participant Multi-Class Vertical Federated Learning
• FedMix: Approximation of Mixup under Mean Augmented Federated Learning
• A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression
• Federated Learning with Only Positive Labels
• Quantifying the Performance of Federated Transfer Learning
• FedFMC: Sequential Efficient Federated Learning on Non-iid Data
• PV-NAS: Practical Neural Architecture Search for Video Recognition
• Distributed Additive Encryption and Quantization for Privacy Preserving Federated Deep Learning
• An Introduction to Communication Efficient Edge Machine Learning