Federated Learning with Cooperating Devices: A Consensus Approach for Massive IoT Networks

Abstract

Federated learning (FL) is emerging as a new paradigm to train machine learning models in distributed systems. Rather than sharing, and disclosing, the training dataset with the server, the model parameters (e.g. neural networks weights and biases) are optimized collectively by large populations of interconnected devices, acting as local learners. FL can be applied to power-constrained IoT devices with slow and sporadic connections. In addition, it does not need data to be exported to third parties, preserving privacy. Despite these benefits, a main limit of existing approaches is the centralized optimization which relies on a server for aggregation and fusion of local parameters; this has the drawback of a single point of failure and scaling issues for increasing network size. The paper proposes a fully distributed (or server-less) learning approach: the proposed FL algorithms leverage the cooperation of devices that perform data operations inside the network by iterating local computations and mutual interactions via consensus-based methods. The approach lays the groundwork for integration of FL within 5G and beyond networks characterized by decentralized connectivity and computing, with intelligence distributed over the end-devices. The proposed methodology is verified by experimental datasets collected inside an industrial IoT environment.

arXiv commentThis work received support from the CHIST-ERA III Grant RadioSense (Big Data and Process Modelling for the Smart Industry - BDSI). The paper has been accepted for publication in the IEEE Internet of Things Journal. The current arXiv contains an additional Appendix C that describes the database and the Python scripts. Published version: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8950073&isnumber=6702522

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Diffusion Adaptation Strategies for Distributed Optimization and Learning over Networks
• Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
• Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
• BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning
• Peer-to-peer Federated Learning on Graphs
• Decentralized Federated Learning: A Segmented Gossip Approach
• GossipGraD: Scalable Deep Learning using Gossip Communication based Asynchronous Gradient Descent
• Gossip training for deep learning
• A cloud-IoT platform for passive radio sensing: challenges and application case studies

Cited by in corpus (38)

• Federated Learning for Internet of Things: A Comprehensive Survey
• Opportunities of Federated Learning in Connected, Cooperative and Automated Industrial Systems
• A Systematic Literature Review on Federated Machine Learning: From A Software Engineering Perspective
• Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks
• Edge Intelligence: Architectures, Challenges, and Applications
• Fusion of Probability Density Functions
• Topology-aware Federated Learning in Edge Computing: A Comprehensive Survey
• Federated learning: Applications, challenges and future directions
• Study of the performance and scalability of federated learning for medical imaging with intermittent clients
• Asynchronous Decentralized Federated Learning for Collaborative Fault Diagnosis of PV Stations
• Federated Learning for Big Data: A Survey on Opportunities, Applications, and Future Directions
• Confederated Learning: Federated Learning with Decentralized Edge Servers
• From Server-Based to Client-Based Machine Learning: A Comprehensive Survey
• Decentralized and Model-Free Federated Learning: Consensus-Based Distillation in Function Space
• Adaptive Federated Learning and Digital Twin for Industrial Internet of Things
• Mobility-Aware Decentralized Federated Learning with Joint Optimization of Local Iteration and Leader Selection for Vehicular Networks
• p2pGNN: A Decentralized Graph Neural Network for Node Classification in Peer-to-Peer Networks
• Federated Learning over Wireless Device-to-Device Networks: Algorithms and Convergence Analysis
• Stochastic Calibration of Radio Interferometers
• Decentralized Federated Learning via SGD over Wireless D2D Networks
• Consensus-Based Distributed Computation of Link-Based Network Metrics
• Proof of Swarm Based Ensemble Learning for Federated Learning Applications
• FLAGS Framework for Comparative Analysis of Federated Learning Algorithms
• Efficient Ring-topology Decentralized Federated Learning with Deep Generative Models for Industrial Artificial Intelligent
• Fog enabled distributed training architecture for federated learning
• Realistic Urban Traffic Generator using Decentralized Federated Learning for the SUMO simulator
• Robustness of Decentralised Learning to Nodes and Data Disruption
• Game Theory and Machine Learning in UAVs-Assisted Wireless Communication Networks: A Survey
• An Energy and Carbon Footprint Analysis of Distributed and Federated Learning
• On the Importance of Trust in Next-Generation Networked CPS Systems: An AI Perspective
• Communication-Efficient Distributed SGD using Preamble-based Random Access
• Real-Time Resource Allocation for Wireless Powered Multiuser Mobile Edge Computing With Energy and Task Causality
• The Effect of Training Parameters and Mechanisms on Decentralized Federated Learning based on MNIST Dataset
• A Framework for Energy and Carbon Footprint Analysis of Distributed and Federated Edge Learning
• Detection of Insider Attacks in Distributed Projected Subgradient Algorithms
• Budgeted Online Selection of Candidate IoT Clients to Participate in Federated Learning
• Dual-Locking Learned AI Models: A PIN-Based Sparse QIM Watermarking and Adaptive Index Permutation Approach
• Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications