Peer-to-peer Federated Learning on Graphs

Abstract

We consider the problem of training a machine learning model over a network of nodes in a fully decentralized framework. The nodes take a Bayesian-like approach via the introduction of a belief over the model parameter space. We propose a distributed learning algorithm in which nodes update their belief by aggregate information from their one-hop neighbors to learn a model that best fits the observations over the entire network. In addition, we also obtain sufficient conditions to ensure that the probability of error is small for every node in the network. We discuss approximations required for applying this algorithm to train Deep Neural Networks (DNNs). Experiments on training linear regression model and on training a DNN show that the proposed learning rule algorithm provides a significant improvement in the accuracy compared to the case where nodes learn without cooperation.

References in corpus (4)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Weight Uncertainty in Neural Networks

Cited by in corpus (30)

• Federated Learning with Cooperating Devices: A Consensus Approach for Massive IoT Networks
• From Distributed Machine Learning to Federated Learning: A Survey
• FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
• A Decentralized Federated Learning Framework via Committee Mechanism with Convergence Guarantee
• Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learning
• Federated Learning for Healthcare Informatics
• Federated Graph Classification over Non-IID Graphs
• Communication Efficiency in Federated Learning: Achievements and Challenges
• Throughput-Optimal Topology Design for Cross-Silo Federated Learning
• Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matching
• Personalized and privacy-preserving federated heterogeneous medical image analysis with PPPML-HMI
• Confederated Learning: Federated Learning with Decentralized Edge Servers
• Hierarchical Federated Learning through LAN-WAN Orchestration
• Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
• Decentralized and Model-Free Federated Learning: Consensus-Based Distillation in Function Space
• BAFFLE : Blockchain Based Aggregator Free Federated Learning
• Learning, Computing, and Trustworthiness in Intelligent IoT Environments: Performance-Energy Tradeoffs
• Communication-Efficient Training Workload Balancing for Decentralized Multi-Agent Learning
• Source Free Unsupervised Graph Domain Adaptation
• A(DP)${}^2$SGD: Asynchronous Decentralized Parallel Stochastic Gradient Descent with Differential Privacy
• Network-Aware Optimization of Distributed Learning for Fog Computing
• Trends and Advancements in Deep Neural Network Communication
• Efficient and Less Centralized Federated Learning
• A Distributed Cubic-Regularized Newton Method for Smooth Convex Optimization over Networks
• Game Theory and Machine Learning in UAVs-Assisted Wireless Communication Networks: A Survey
• Separation of Powers in Federated Learning
• Communication-efficient Decentralized Machine Learning over Heterogeneous Networks
• PPT: A Privacy-Preserving Global Model Training Protocol for Federated Learning in P2P Networks
• The Effect of Training Parameters and Mechanisms on Decentralized Federated Learning based on MNIST Dataset
• MAcPNN: Mutual Assisted Learning on Data Streams with Temporal Dependence