A Survey on Collaborative DNN Inference for Edge Intelligence

Abstract

With the vigorous development of artificial intelligence (AI), the intelligent applications based on deep neural network (DNN) change people's lifestyles and the production efficiency. However, the huge amount of computation and data generated from the network edge becomes the major bottleneck, and traditional cloud-based computing mode has been unable to meet the requirements of real-time processing tasks. To solve the above problems, by embedding AI model training and inference capabilities into the network edge, edge intelligence (EI) becomes a cutting-edge direction in the field of AI. Furthermore, collaborative DNN inference among the cloud, edge, and end device provides a promising way to boost the EI. Nevertheless, at present, EI oriented collaborative DNN inference is still in its early stage, lacking a systematic classification and discussion of existing research efforts. Thus motivated, we have made a comprehensive investigation on the recent studies about EI oriented collaborative DNN inference. In this paper, we firstly review the background and motivation of EI. Then, we classify four typical collaborative DNN inference paradigms for EI, and analyze the characteristics and key technologies of them. Finally, we summarize the current challenges of collaborative DNN inference, discuss the future development trend and provide the future research direction.

References in corpus (13)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SSD: Single Shot MultiBox Detector
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence
• SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud
• NestDNN: Resource-Aware Multi-Tenant On-Device Deep Learning for Continuous Mobile Vision
• Multi-Robot Collaborative Perception with Graph Neural Networks
• UAVs as a Service: Boosting Edge Intelligence for Air-Ground Integrated Networks
• An Introduction to Probabilistic Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications
• Distributed CNN Inference on Resource-Constrained UAVs for Surveillance Systems: Design and Optimization
• Distributed Deep Convolutional Neural Networks for the Internet-of-Things
• Guardians of the Deep Fog: Failure-Resilient DNN Inference from Edge to Cloud
• ResiliNet: Failure-Resilient Inference in Distributed Neural Networks

Cited by in corpus (4)

• Semantic Edge Computing and Semantic Communications in 6G Networks: A Unifying Survey and Research Challenges
• GCoDE: Efficient Device-Edge Co-Inference for GNNs via Architecture-Mapping Co-Search
• Improving inference time in multi-TPU systems with profiled model segmentation
• Edge Intelligence in Civil Aviation: Paradigms, Techniques, and Applications