Edge-assisted Democratized Learning Towards Federated Analytics

Abstract

A recent take towards Federated Analytics (FA), which allows analytical insights of distributed datasets, reuses the Federated Learning (FL) infrastructure to evaluate the summary of model performances across the training devices. However, the current realization of FL adopts single server-multiple client architecture with limited scope for FA, which often results in learning models with poor generalization, i.e., an ability to handle new/unseen data, for real-world applications. Moreover, a hierarchical FL structure with distributed computing platforms demonstrates incoherent model performances at different aggregation levels. Therefore, we need to design a robust learning mechanism than the FL that (i) unleashes a viable infrastructure for FA and (ii) trains learning models with better generalization capability. In this work, we adopt the novel democratized learning (Dem-AI) principles and designs to meet these objectives. Firstly, we show the hierarchical learning structure of the proposed edge-assisted democratized learning mechanism, namely Edge-DemLearn, as a practical framework to empower generalization capability in support of FA. Secondly, we validate Edge-DemLearn as a flexible model training mechanism to build a distributed control and aggregation methodology in regions by leveraging the distributed computing infrastructure. The distributed edge computing servers construct regional models, minimize the communication loads, and ensure distributed data analytic application's scalability. To that end, we adhere to a near-optimal two-sided many-to-one matching approach to handle the combinatorial constraints in Edge-DemLearn and solve it for fast knowledge acquisition with optimization of resource allocation and associations between multiple servers and devices. Extensive simulation results on real datasets demonstrate the effectiveness of the proposed methods.

arXiv commentAccepted for publication in IEEE Internet of Things Journal

References in corpus (15)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge
• Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• CVXPY: A Python-Embedded Modeling Language for Convex Optimization
• Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocation
• Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent
• Personalized Federated Learning: A Meta-Learning Approach
• Adaptive Personalized Federated Learning
• A Crowdsourcing Framework for On-Device Federated Learning
• LEAF: A Benchmark for Federated Settings
• An Incentive Mechanism for Federated Learning in Wireless Cellular network: An Auction Approach
• Self-organizing Democratized Learning: Towards Large-scale Distributed Learning Systems

Cited by in corpus (3)

• Optimal Resource Management for Hierarchical Federated Learning over HetNets with Wireless Energy Transfer
• A Contribution-based Device Selection Scheme in Federated Learning
• From Learning to Analytics: Improving Model Efficacy with Goal-Directed Client Selection