Federated Learning on Non-IID Data: A Survey

Abstract

Federated learning is an emerging distributed machine learning framework for privacy preservation. However, models trained in federated learning usually have worse performance than those trained in the standard centralized learning mode, especially when the training data are not independent and identically distributed (Non-IID) on the local devices. In this survey, we pro-vide a detailed analysis of the influence of Non-IID data on both parametric and non-parametric machine learning models in both horizontal and vertical federated learning. In addition, cur-rent research work on handling challenges of Non-IID data in federated learning are reviewed, and both advantages and disadvantages of these approaches are discussed. Finally, we suggest several future research directions before concluding the paper.

References in corpus (66)

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• Overcoming catastrophic forgetting in neural networks
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• mixup: Beyond Empirical Risk Minimization
• Knowledge Distillation: A Survey
• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Learning with Non-IID Data
• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• Towards Federated Learning at Scale: System Design
• Secure Federated Transfer Learning
• Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation
• Federated Learning with Personalization Layers
• Differentially Private Federated Learning: A Client Level Perspective
• FedMD: Heterogenous Federated Learning via Model Distillation
• EMNIST: an extension of MNIST to handwritten letters
• Federated Optimization in Heterogeneous Networks
• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• iDLG: Improved Deep Leakage from Gradients
• Improving Federated Learning Personalization via Model Agnostic Meta Learning
• FedML: A Research Library and Benchmark for Federated Machine Learning
• Adaptive Personalized Federated Learning
• Three Approaches for Personalization with Applications to Federated Learning
• HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning
• Federated Meta-Learning with Fast Convergence and Efficient Communication
• Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems
• Federated Learning of a Mixture of Global and Local Models
• Threats to Federated Learning: A Survey
• SecureBoost: A Lossless Federated Learning Framework
• On the Convergence of Local Descent Methods in Federated Learning
• Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
• Measuring the Effects of Data Parallelism on Neural Network Training
• Federated Adversarial Domain Adaptation
• Robust Federated Learning in a Heterogeneous Environment
• Overcoming Forgetting in Federated Learning on Non-IID Data
• Think Locally, Act Globally: Federated Learning with Local and Global Representations
• Adaptive Federated Optimization
• Variational Federated Multi-Task Learning
• Federated Learning with Matched Averaging
• A Framework for Evaluating Gradient Leakage Attacks in Federated Learning
• Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator
• Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer
• VAFL: a Method of Vertical Asynchronous Federated Learning
• Federated Learning on Non-IID Data Silos: An Experimental Study
• Multi-Participant Multi-Class Vertical Federated Learning
• Entity Resolution and Federated Learning get a Federated Resolution
• Real-World Image Datasets for Federated Learning
• Salvaging Federated Learning by Local Adaptation
• FedMix: Approximation of Mixup under Mean Augmented Federated Learning
• A Communication Efficient Collaborative Learning Framework for Distributed Features
• A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression
• No Peek: A Survey of private distributed deep learning
• Federated Learning with Only Positive Labels
• Personalized Cross-Silo Federated Learning on Non-IID Data
• The Tradeoff Between Privacy and Accuracy in Anomaly Detection Using Federated XGBoost
• Federated Neural Architecture Search
• Quantifying the Performance of Federated Transfer Learning
• FedFMC: Sequential Efficient Federated Learning on Non-iid Data
• Self-supervised Cross-silo Federated Neural Architecture Search
• Hybrid-FL for Wireless Networks: Cooperative Learning Mechanism Using Non-IID Data
• Improving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models
• Practical One-Shot Federated Learning for Cross-Silo Setting
• PV-NAS: Practical Neural Architecture Search for Video Recognition
• Distributed Additive Encryption and Quantization for Privacy Preserving Federated Deep Learning
• An End-to-End Encrypted Neural Network for Gradient Updates Transmission in Federated Learning
• An Introduction to Communication Efficient Edge Machine Learning