New Directions in Distributed Deep Learning: Bringing the Network at Forefront of IoT Design

Abstract

In this paper, we first highlight three major challenges to large-scale adoption of deep learning at the edge: (i) Hardware-constrained IoT devices, (ii) Data security and privacy in the IoT era, and (iii) Lack of network-aware deep learning algorithms for distributed inference across multiple IoT devices. We then provide a unified view targeting three research directions that naturally emerge from the above challenges: (1) Federated learning for training deep networks, (2) Data-independent deployment of learning algorithms, and (3) Communication-aware distributed inference. We believe that the above research directions need a network-centric approach to enable the edge intelligence and, therefore, fully exploit the true potential of IoT.

This preprint is for personal use only. The official article will appear in proceedings of Design Automation Conference (DAC), 2020. This work was presented at the DAC 2020 special session on Edge-to-Cloud Neural Networks for Machine Learning Applications in Future IoT Systems

References in corpus (10)

• Distilling the Knowledge in a Neural Network
• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Optimization:Distributed Optimization Beyond the Datacenter
• LEAF: A Benchmark for Federated Settings
• Data-Free Knowledge Distillation for Deep Neural Networks
• Federated Learning with Bayesian Differential Privacy
• Layer-compensated Pruning for Resource-constrained Convolutional Neural Networks
• Knowledge distillation using unlabeled mismatched images
• FedMAX: Mitigating Activation Divergence for Accurate and Communication-Efficient Federated Learning
• EdgeAI: A Vision for Deep Learning in IoT Era