Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications

Abstract

Deep neural networks (DNNs) have achieved unprecedented success in the field of artificial intelligence (AI), including computer vision, natural language processing and speech recognition. However, their superior performance comes at the considerable cost of computational complexity, which greatly hinders their applications in many resource-constrained devices, such as mobile phones and Internet of Things (IoT) devices. Therefore, methods and techniques that are able to lift the efficiency bottleneck while preserving the high accuracy of DNNs are in great demand in order to enable numerous edge AI applications. This paper provides an overview of efficient deep learning methods, systems and applications. We start from introducing popular model compression methods, including pruning, factorization, quantization as well as compact model design. To reduce the large design cost of these manual solutions, we discuss the AutoML framework for each of them, such as neural architecture search (NAS) and automated pruning and quantization. We then cover efficient on-device training to enable user customization based on the local data on mobile devices. Apart from general acceleration techniques, we also showcase several task-specific accelerations for point cloud, video and natural language processing by exploiting their spatial sparsity and temporal/token redundancy. Finally, to support all these algorithmic advancements, we introduce the efficient deep learning system design from both software and hardware perspectives.

Journal preprint (ACM TODAES, 2021). The first seven authors contributed equally to this work and are listed in the alphabetical order

References in corpus (33)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Distilling the Knowledge in a Neural Network
• Sequence to Sequence Learning with Neural Networks
• Two-Stream Convolutional Networks for Action Recognition in Videos
• Neural Architecture Search with Reinforcement Learning
• FitNets: Hints for Thin Deep Nets
• Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
• Deep Learning with Limited Numerical Precision
• FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
• Compressing Deep Convolutional Networks using Vector Quantization
• Linformer: Self-Attention with Linear Complexity
• O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis
• Trained Ternary Quantization
• Spatiotemporal Residual Networks for Video Action Recognition
• SMASH: One-Shot Model Architecture Search through HyperNetworks
• Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
• Slimmable Neural Networks
• Device Placement Optimization with Reinforcement Learning
• PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning
• A Survey on Neural Architecture Search
• Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
• AutoSlim: Towards One-Shot Architecture Search for Channel Numbers
• SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks
• SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers
• Kaolin: A PyTorch Library for Accelerating 3D Deep Learning Research
• PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution
• TorchSparse: Efficient Point Cloud Inference Engine
• AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations
• Hardware-Centric AutoML for Mixed-Precision Quantization
• A3D: Adaptive 3D Networks for Video Action Recognition
• Training Kinetics in 15 Minutes: Large-scale Distributed Training on Videos
• PV-NAS: Practical Neural Architecture Search for Video Recognition