End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things

Abstract

This work is the first attempt to evaluate and compare felderated learning (FL) and split neural networks (SplitNN) in real-world IoT settings in terms of learning performance and device implementation overhead. We consider a variety of datasets, different model architectures, multiple clients, and various performance metrics. For learning performance, which is specified by the model accuracy and convergence speed metrics, we empirically evaluate both FL and SplitNN under different types of data distributions such as imbalanced and non-independent and identically distributed (non-IID) data. We show that the learning performance of SplitNN is better than FL under an imbalanced data distribution, but worse than FL under an extreme non-IID data distribution. For implementation overhead, we end-to-end mount both FL and SplitNN on Raspberry Pis, and comprehensively evaluate overheads including training time, communication overhead under the real LAN setting, power consumption and memory usage. Our key observations are that under IoT scenario where the communication traffic is the main concern, the FL appears to perform better over SplitNN because FL has the significantly lower communication overhead compared with SplitNN, which empirically corroborate previous statistical analysis. In addition, we reveal several unrecognized limitations about SplitNN, forming the basis for future research.

arXiv comment10 pages, 12 figures

References in corpus (9)

• Distilling the Knowledge in a Neural Network
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning with Non-IID Data
• On the Convergence of FedAvg on Non-IID Data
• No Peek: A Survey of private distributed deep learning
• Detailed comparison of communication efficiency of split learning and federated learning
• Training wide residual networks for deployment using a single bit for each weight
• Federated Continual Learning with Weighted Inter-client Transfer

Cited by in corpus (6)

• Precision Health Data: Requirements, Challenges and Existing Techniques for Data Security and Privacy
• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• Communication and Computation Reduction for Split Learning using Asynchronous Training
• Federated Split Vision Transformer for COVID-19 CXR Diagnosis using Task-Agnostic Training
• Towards Representation Identical Privacy-Preserving Graph Neural Network via Split Learning
• ASFGNN: Automated Separated-Federated Graph Neural Network