Bringing AI To Edge: From Deep Learning's Perspective

Abstract

Edge computing and artificial intelligence (AI), especially deep learning for nowadays, are gradually intersecting to build a novel system, called edge intelligence. However, the development of edge intelligence systems encounters some challenges, and one of these challenges is the \textit{computational gap} between computation-intensive deep learning algorithms and less-capable edge systems. Due to the computational gap, many edge intelligence systems cannot meet the expected performance requirements. To bridge the gap, a plethora of deep learning techniques and optimization methods are proposed in the past years: light-weight deep learning models, network compression, and efficient neural architecture search. Although some reviews or surveys have partially covered this large body of literature, we lack a systematic and comprehensive review to discuss all aspects of these deep learning techniques which are critical for edge intelligence implementation. As various and diverse methods which are applicable to edge systems are proposed intensively, a holistic review would enable edge computing engineers and community to know the state-of-the-art deep learning techniques which are instrumental for edge intelligence and to facilitate the development of edge intelligence systems. This paper surveys the representative and latest deep learning techniques that are useful for edge intelligence systems, including hand-crafted models, model compression, hardware-aware neural architecture search and adaptive deep learning models. Finally, based on observations and simple experiments we conducted, we discuss some future directions.

References in corpus (67)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SSD: Single Shot MultiBox Detector
• Distilling the Knowledge in a Neural Network
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• YOLOv3: An Incremental Improvement
• Neural Architecture Search with Reinforcement Learning
• Categorical Reparameterization with Gumbel-Softmax
• End to End Learning for Self-Driving Cars
• Mobile Edge Computing: A Survey on Architecture and Computation Offloading
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• MobileNetV2: Inverted Residuals and Linear Bottlenecks
• FitNets: Hints for Thin Deep Nets
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Wide Residual Networks
• Neural Architecture Search: A Survey
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• Deep Learning with Limited Numerical Precision
• In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning
• Rethinking the Value of Network Pruning
• Ternary Weight Networks
• Trained Ternary Quantization
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
• A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Caching at the Wireless Edge: Design Aspects, Challenges and Future Directions
• Towards Accurate Binary Convolutional Neural Network
• Channel Pruning for Accelerating Very Deep Neural Networks
• Adaptive Computation Time for Recurrent Neural Networks
• ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
• Light-Head R-CNN: In Defense of Two-Stage Object Detector
• MCUNet: Tiny Deep Learning on IoT Devices
• Model compression via distillation and quantization
• CondConv: Conditionally Parameterized Convolutions for Efficient Inference
• Slimmable Neural Networks
• Adaptive Multi-Teacher Multi-level Knowledge Distillation
• Dynamic Channel Pruning: Feature Boosting and Suppression
• Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference
• Discrimination-aware Channel Pruning for Deep Neural Networks
• Channel Gating Neural Networks
• Lite Transformer with Long-Short Range Attention
• GhostNet: More Features from Cheap Operations
• Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks
• DSOD: Learning Deeply Supervised Object Detectors from Scratch
• MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
• SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers
• Feature-map-level Online Adversarial Knowledge Distillation
• What is the State of Neural Network Pruning?
• Proving the Lottery Ticket Hypothesis: Pruning is All You Need
• NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications
• ZynqNet: An FPGA-Accelerated Embedded Convolutional Neural Network
• Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networks
• On-Device Machine Learning: An Algorithms and Learning Theory Perspective
• LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
• Tiny SSD: A Tiny Single-shot Detection Deep Convolutional Neural Network for Real-time Embedded Object Detection
• DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework
• Defensive Quantization: When Efficiency Meets Robustness
• TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning
• Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm
• APQ: Joint Search for Network Architecture, Pruning and Quantization Policy
• SBNet: Sparse Blocks Network for Fast Inference
• Densely Connected Search Space for More Flexible Neural Architecture Search
• MnasFPN: Learning Latency-aware Pyramid Architecture for Object Detection on Mobile Devices
• Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search
• Resource Constrained Neural Network Architecture Search: Will a Submodularity Assumption Help?
• Enabling On-Device CNN Training by Self-Supervised Instance Filtering and Error Map Pruning

Cited by in corpus (4)

• Synthetic data generation method for data-free knowledge distillation in regression neural networks
• No Free Lunch: Balancing Learning and Exploitation at the Network Edge
• HSCoNAS: Hardware-Software Co-Design of Efficient DNNs via Neural Architecture Search
• Toward Compact Parameter Representations for Architecture-Agnostic Neural Network Compression