A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection

Abstract

Federated learning has been a hot research topic in enabling the collaborative training of machine learning models among different organizations under the privacy restrictions. As researchers try to support more machine learning models with different privacy-preserving approaches, there is a requirement in developing systems and infrastructures to ease the development of various federated learning algorithms. Similar to deep learning systems such as PyTorch and TensorFlow that boost the development of deep learning, federated learning systems (FLSs) are equivalently important, and face challenges from various aspects such as effectiveness, efficiency, and privacy. In this survey, we conduct a comprehensive review on federated learning systems. To achieve smooth flow and guide future research, we introduce the definition of federated learning systems and analyze the system components. Moreover, we provide a thorough categorization for federated learning systems according to six different aspects, including data distribution, machine learning model, privacy mechanism, communication architecture, scale of federation and motivation of federation. The categorization can help the design of federated learning systems as shown in our case studies. By systematically summarizing the existing federated learning systems, we present the design factors, case studies, and future research opportunities.

Accepted to IEEE Transactions on Knowledge and Data Engineering (TKDE)

References in corpus (50)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Distilling the Knowledge in a Neural Network
• Federated Learning: Challenges, Methods, and Future Directions
• An Overview of Multi-Task Learning in Deep Neural Networks
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• On the Convergence of FedAvg on Non-IID Data
• Towards Federated Learning at Scale: System Design
• Secure Federated Transfer Learning
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• Secure Federated Matrix Factorization
• Applied Federated Learning: Improving Google Keyboard Query Suggestions
• Federated Optimization in Heterogeneous Networks
• Asynchronous Federated Optimization
• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• Can You Really Backdoor Federated Learning?
• Improving Federated Learning Personalization via Model Agnostic Meta Learning
• FedML: A Research Library and Benchmark for Federated Machine Learning
• HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning
• LEAF: A Benchmark for Federated Settings
• Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems
• Agnostic Federated Learning
• Protection Against Reconstruction and Its Applications in Private Federated Learning
• Federated Learning of a Mixture of Global and Local Models
• Threats to Federated Learning: A Survey
• Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
• Inverting Gradients -- How easy is it to break privacy in federated learning?
• Astraea: Self-balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applications
• Practical Secure Aggregation for Federated Learning on User-Held Data
• SecureBoost: A Lossless Federated Learning Framework
• Split learning for health: Distributed deep learning without sharing raw patient data
• Differential Privacy-enabled Federated Learning for Sensitive Health Data
• Data Poisoning Attacks on Factorization-Based Collaborative Filtering
• Federated Forest
• Adaptive Federated Optimization
• Variational Federated Multi-Task Learning
• Performance Optimization for Federated Person Re-identification via Benchmark Analysis
• Differentially Private Learning with Adaptive Clipping
• Real-World Image Datasets for Federated Learning
• Lower Bounds and Optimal Algorithms for Personalized Federated Learning
• On Lightweight Privacy-Preserving Collaborative Learning for IoT Objects
• Boosting Privately: Privacy-Preserving Federated Extreme Boosting for Mobile Crowdsensing
• The OARF Benchmark Suite: Characterization and Implications for Federated Learning Systems
• Local Differential Privacy based Federated Learning for Internet of Things
• Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approach
• FedEval: A Holistic Evaluation Framework for Federated Learning
• Adaptive Kernel Value Caching for SVM Training
• Evaluation Framework For Large-scale Federated Learning

Cited by in corpus (40)

• From Distributed Machine Learning to Federated Learning: A Survey
• Vertical Federated Learning: Concepts, Advances and Challenges
• Learning to Detect Malicious Clients for Robust Federated Learning
• Privacy-preserving Federated Learning for Residential Short Term Load Forecasting
• Flexible Clustered Federated Learning for Client-Level Data Distribution Shift
• Topology-aware Federated Learning in Edge Computing: A Comprehensive Survey
• Privacy-preserving Artificial Intelligence Techniques in Biomedicine
• AI Security for Geoscience and Remote Sensing: Challenges and Future Trends
• Federated Learning on Non-IID Data Silos: An Experimental Study
• Privacy-Preserving Machine Learning: Methods, Challenges and Directions
• Federated Multi-view Matrix Factorization for Personalized Recommendations
• Communication Efficiency in Federated Learning: Achievements and Challenges
• The OARF Benchmark Suite: Characterization and Implications for Federated Learning Systems
• Federated Transfer-Ordered-Personalized Learning for Driver Monitoring Application
• Towards Ubiquitous AI in 6G with Federated Learning
• A Survey on Federated Learning and its Applications for Accelerating Industrial Internet of Things
• Learn Electronic Health Records by Fully Decentralized Federated Learning
• An Experimental Study of Class Imbalance in Federated Learning
• Supporting AI Engineering on the IoT Edge through Model-Driven TinyML
• Ternary Compression for Communication-Efficient Federated Learning
• Byzantine-Robust and Privacy-Preserving Framework for FedML
• Federated Learning System without Model Sharing through Integration of Dimensional Reduced Data Representations
• Sample-based and Feature-based Federated Learning for Unconstrained and Constrained Nonconvex Optimization via Mini-batch SSCA
• DistFL: Distribution-aware Federated Learning for Mobile Scenarios
• Tackling Dynamics in Federated Incremental Learning with Variational Embedding Rehearsal
• Federated Transfer Learning with Dynamic Gradient Aggregation
• Federated Learning on Non-IID Data: A Survey
• SoK: Training Machine Learning Models over Multiple Sources with Privacy Preservation
• Benchmarking Differential Privacy and Federated Learning for BERT Models
• FederatedNILM: A Distributed and Privacy-preserving Framework for Non-intrusive Load Monitoring based on Federated Deep Learning
• Accuracy and Privacy Evaluations of Collaborative Data Analysis
• Dubhe: Towards Data Unbiasedness with Homomorphic Encryption in Federated Learning Client Selection
• Differentially Private Federated Learning via Inexact ADMM
• Meta-learning Amidst Heterogeneity and Ambiguity
• Fault-Tolerant Federated Reinforcement Learning with Theoretical Guarantee
• Simeon -- Secure Federated Machine Learning Through Iterative Filtering
• Management of Resource at the Network Edge for Federated Learning
• Sensing and Mapping for Better Roads: Initial Plan for Using Federated Learning and Implementing a Digital Twin to Identify the Road Conditions in a Developing Country -- Sri Lanka
• Budgeted Online Selection of Candidate IoT Clients to Participate in Federated Learning
• Dynamic Gradient Aggregation for Federated Domain Adaptation