26ms Inference Time for ResNet-50: Towards Real-Time Execution of all DNNs on Smartphone
arXiv:1905.00571
Abstract
With the rapid emergence of a spectrum of high-end mobile devices, many applications that required desktop-level computation capability formerly can now run on these devices without any problem. However, without a careful optimization, executing Deep Neural Networks (a key building block of the real-time video stream processing that is the foundation of many popular applications) is still challenging, specifically, if an extremely low latency or high accuracy inference is needed. This work presents CADNN, a programming framework to efficiently execute DNN on mobile devices with the help of advanced model compression (sparsity) and a set of thorough architecture-aware optimization. The evaluation result demonstrates that CADNN outperforms all the state-of-the-art dense DNN execution frameworks like TensorFlow Lite and TVM.
References in corpus (4)
- Binarized Neural Networks
- A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers
- Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
- Progressive DNN Compression: A Key to Achieve Ultra-High Weight Pruning and Quantization Rates using ADMM
Cited by in corpus (6)
- Tiny but Accurate: A Pruned, Quantized and Optimized Memristor Crossbar Framework for Ultra Efficient DNN Implementation
- Teachers Do More Than Teach: Compressing Image-to-Image Models
- An Ultra-Efficient Memristor-Based DNN Framework with Structured Weight Pruning and Quantization Using ADMM
- Acting in Delayed Environments with Non-Stationary Markov Policies
- GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity
- A Novel ANN Structure for Image Recognition