Resource-Constrained Federated Learning with Heterogeneous Labels and Models

Abstract

Various IoT applications demand resource-constrained machine learning mechanisms for different applications such as pervasive healthcare, activity monitoring, speech recognition, real-time computer vision, etc. This necessitates us to leverage information from multiple devices with few communication overheads. Federated Learning proves to be an extremely viable option for distributed and collaborative machine learning. Particularly, on-device federated learning is an active area of research, however, there are a variety of challenges in addressing statistical (non-IID data) and model heterogeneities. In addition, in this paper we explore a new challenge of interest -- to handle label heterogeneities in federated learning. To this end, we propose a framework with simple $\alpha$-weighted federated aggregation of scores which leverages overlapping information gain across labels, while saving bandwidth costs in the process. Empirical evaluation on Animals-10 dataset (with 4 labels for effective elucidation of results) indicates an average deterministic accuracy increase of at least ~16.7%. We also demonstrate the on-device capabilities of our proposed framework by experimenting with federated learning and inference across different iterations on a Raspberry Pi 2, a single-board computing platform.

6 pages, 5 figures, ACM KDD 2020 (The 3rd International Workshop on Artificial Intelligence of Things - AIoT'20)

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Learning with Non-IID Data
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• FedMD: Heterogenous Federated Learning via Model Distillation
• Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
• Adaptive Personalized Federated Learning
• Robust Federated Learning in a Heterogeneous Environment

Cited by in corpus (2)

• Federated Learning for Smart Healthcare: A Survey
• Federated Learning with Heterogeneous Labels and Models for Mobile Activity Monitoring