The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning

Abstract

The Internet of Things (IoT) is on the verge of a major paradigm shift. In the IoT system of the future, IoFT, the cloud will be substituted by the crowd where model training is brought to the edge, allowing IoT devices to collaboratively extract knowledge and build smart analytics/models while keeping their personal data stored locally. This paradigm shift was set into motion by the tremendous increase in computational power on IoT devices and the recent advances in decentralized and privacy-preserving model training, coined as federated learning (FL). This article provides a vision for IoFT and a systematic overview of current efforts towards realizing this vision. Specifically, we first introduce the defining characteristics of IoFT and discuss FL data-driven approaches, opportunities, and challenges that allow decentralized inference within three dimensions: (i) a global model that maximizes utility across all IoT devices, (ii) a personalized model that borrows strengths across all devices yet retains its own model, (iii) a meta-learning model that quickly adapts to new devices or learning tasks. We end by describing the vision and challenges of IoFT in reshaping different industries through the lens of domain experts. Those industries include manufacturing, transportation, energy, healthcare, quality & reliability, business, and computing.

arXiv commentAccepted at IEEE

References in corpus (82)

• Adam: A Method for Stochastic Optimization
• Overcoming catastrophic forgetting in neural networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Federated Learning: Challenges, Methods, and Future Directions
• Variational Inference: A Review for Statisticians
• Federated Learning: Strategies for Improving Communication Efficiency
• Equality of Opportunity in Supervised Learning
• Federated Learning with Non-IID Data
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• A continual learning survey: Defying forgetting in classification tasks
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Federated Learning for Mobile Keyboard Prediction
• In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• How To Backdoor Federated Learning
• SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
• Consistency of the group Lasso and multiple kernel learning
• Federated Learning with Personalization Layers
• Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications
• Applied Federated Learning: Improving Google Keyboard Query Suggestions
• CoLight: Learning Network-level Cooperation for Traffic Signal Control
• COVID-ResNet: A Deep Learning Framework for Screening of COVID19 from Radiographs
• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent
• Communication Efficient Distributed Optimization using an Approximate Newton-type Method
• Improving Federated Learning Personalization via Model Agnostic Meta Learning
• Fair Resource Allocation in Federated Learning
• Adaptive Personalized Federated Learning
• Fairness Constraints: Mechanisms for Fair Classification
• Three Approaches for Personalization with Applications to Federated Learning
• Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
• Prochlo: Strong Privacy for Analytics in the Crowd
• ALEX: An Updatable Adaptive Learned Index
• Federated Meta-Learning with Fast Convergence and Efficient Communication
• Collecting Telemetry Data Privately
• Agnostic Federated Learning
• Protection Against Reconstruction and Its Applications in Private Federated Learning
• Learning Task Grouping and Overlap in Multi-task Learning
• PCC: Re-architecting Congestion Control for Consistent High Performance
• Federated Learning of a Mixture of Global and Local Models
• Threats to Federated Learning: A Survey
• Ditto: Fair and Robust Federated Learning Through Personalization
• Astraea: Self-balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applications
• Sparsity in multiple kernel learning
• Early stopping and non-parametric regression: An optimal data-dependent stopping rule
• A Decomposition Algorithm to Solve the Multi-Hop Peer-to-Peer Ride-Matching Problem
• Federated Learning for Emoji Prediction in a Mobile Keyboard
• Bayesian Model-Agnostic Meta-Learning
• Think Locally, Act Globally: Federated Learning with Local and Global Representations
• A Convex Formulation for Learning Task Relationships in Multi-Task Learning
• Federated Evaluation of On-device Personalization
• DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression
• Federated Learning Of Out-Of-Vocabulary Words
• The Unusual Effectiveness of Averaging in GAN Training
• Focus: Querying Large Video Datasets with Low Latency and Low Cost
• Optimal Client Sampling for Federated Learning
• Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator
• FedSplit: An algorithmic framework for fast federated optimization
• Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies
• Entity Resolution and Federated Learning get a Federated Resolution
• Real-World Image Datasets for Federated Learning
• How do heterogeneities in operating environments affect field failure predictions and test planning?
• Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels
• Personalized and Private Peer-to-Peer Machine Learning
• Training Neural Networks Using Features Replay
• Federated Learning for Ranking Browser History Suggestions
• Minimizing Negative Transfer of Knowledge in Multivariate Gaussian Processes: A Scalable and Regularized Approach
• Survey on Additive Manufacturing, Cloud 3D Printing and Services
• Fairness and Accuracy in Federated Learning
• Improving Fairness via Federated Learning
• The Functional Neural Process
• Real-Time Edge Intelligence in the Making: A Collaborative Learning Framework via Federated Meta-Learning
• Adaptive Deep Kernel Learning
• Space-Filling Designs for Robustness Experiments
• LoAdaBoost: loss-based AdaBoost federated machine learning with reduced computational complexity on IID and non-IID intensive care data
• Meta-learning autoencoders for few-shot prediction
• Uncertainty in Multitask Transfer Learning
• Variational Implicit Processes
• First Order Generative Adversarial Networks

Cited by in corpus (9)

• Generalizing intrusion detection for heterogeneous networks: A stacked-unsupervised federated learning approach
• GIFAIR-FL: A Framework for Group and Individual Fairness in Federated Learning
• Fed-ensemble: Improving Generalization through Model Ensembling in Federated Learning
• Federated Data Analytics: A Study on Linear Models
• Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling
• Collaborative and Distributed Bayesian Optimization via Consensus: Showcasing the Power of Collaboration for Optimal Design
• Differentially Private Distributed Estimation and Learning
• Personalized Tucker Decomposition: Modeling Commonality and Peculiarity on Tensor Data
• Collaborative and Federated Black-box Optimization: A Bayesian Optimization Perspective