Federated Action Recognition on Heterogeneous Embedded Devices

Abstract

Federated learning allows a large number of devices to jointly learn a model without sharing data. In this work, we enable clients with limited computing power to perform action recognition, a computationally heavy task. We first perform model compression at the central server through knowledge distillation on a large dataset. This allows the model to learn complex features and serves as an initialization for model fine-tuning. The fine-tuning is required because the limited data present in smaller datasets is not adequate for action recognition models to learn complex spatio-temporal features. Because the clients present are often heterogeneous in their computing resources, we use an asynchronous federated optimization and we further show a convergence bound. We compare our approach to two baseline approaches: fine-tuning at the central server (no clients) and fine-tuning using (heterogeneous) clients using synchronous federated averaging. We empirically show on a testbed of heterogeneous embedded devices that we can perform action recognition with comparable accuracy to the two baselines above, while our asynchronous learning strategy reduces the training time by 40%, relative to synchronous learning.

arXiv comment13 pages, 12 figures

References in corpus (12)

• Distilling the Knowledge in a Neural Network
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• The Kinetics Human Action Video Dataset
• Deep Visual Domain Adaptation: A Survey
• Towards Federated Learning at Scale: System Design
• Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
• Asynchronous Federated Optimization
• Large scale distributed neural network training through online distillation
• EdgeSpeechNets: Highly Efficient Deep Neural Networks for Speech Recognition on the Edge
• Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints
• Helios: Heterogeneity-Aware Federated Learning with Dynamically Balanced Collaboration

Cited by in corpus (1)

• The Federation Strikes Back: A Survey of Federated Learning Privacy Attacks, Defenses, Applications, and Policy Landscape