A Federated Learning Aggregation Algorithm for Pervasive Computing: Evaluation and Comparison

Abstract

Pervasive computing promotes the installation of connected devices in our living spaces in order to provide services. Two major developments have gained significant momentum recently: an advanced use of edge resources and the integration of machine learning techniques for engineering applications. This evolution raises major challenges, in particular related to the appropriate distribution of computing elements along an edge-to-cloud continuum. About this, Federated Learning has been recently proposed for distributed model training in the edge. The principle of this approach is to aggregate models learned on distributed clients in order to obtain a new, more general model. The resulting model is then redistributed to clients for further training. To date, the most popular federated learning algorithm uses coordinate-wise averaging of the model parameters for aggregation. However, it has been shown that this method is not adapted in heterogeneous environments where data is not identically and independently distributed (non-iid). This corresponds directly to some pervasive computing scenarios where heterogeneity of devices and users challenges machine learning with the double objective of generalization and personalization. In this paper, we propose a novel aggregation algorithm, termed FedDist, which is able to modify its model architecture (here, deep neural network) by identifying dissimilarities between specific neurons amongst the clients. This permits to account for clients' specificity without impairing generalization. Furthermore, we define a complete method to evaluate federated learning in a realistic way taking generalization and personalization into account. Using this method, FedDist is extensively tested and compared with three state-of-the-art federated learning algorithms on the pervasive domain of Human Activity Recognition with smartphones.

arXiv comment9th IEEE International Conference on Pervasive Computing and Communications (PerCom 2021)

References in corpus (10)

• TensorFlow: A system for large-scale machine learning
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Deep Learning for Sensor-based Activity Recognition: A Survey
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Federated Learning with Personalization Layers
• FedMD: Heterogenous Federated Learning via Model Distillation
• Federated Optimization in Heterogeneous Networks
• Bayesian Nonparametric Federated Learning of Neural Networks
• Federated Learning with Matched Averaging
• Arcades: A deep model for adaptive decision making in voice controlled smart-home

Cited by in corpus (6)

• A distillation-based approach integrating continual learning and federated learning for pervasive services
• Threats and Defenses in Federated Learning Life Cycle: A Comprehensive Survey and Challenges
• Evaluation and comparison of federated learning algorithms for Human Activity Recognition on smartphones
• Modeling and Analysis of Three Properties of Mobile Interactive Systems Based on Variable Petri Nets
• Federated Self-Supervised Learning in Heterogeneous Settings: Limits of a Baseline Approach on HAR
• Federated Learning and catastrophic forgetting in pervasive computing: demonstration in HAR domain