CoCoPIE: Making Mobile AI Sweet As PIE --Compression-Compilation Co-Design Goes a Long Way
arXiv:2003.06700
Abstract
Assuming hardware is the major constraint for enabling real-time mobile intelligence, the industry has mainly dedicated their efforts to developing specialized hardware accelerators for machine learning and inference. This article challenges the assumption. By drawing on a recent real-time AI optimization framework CoCoPIE, it maintains that with effective compression-compiler co-design, it is possible to enable real-time artificial intelligence on mainstream end devices without special hardware. CoCoPIE is a software framework that holds numerous records on mobile AI: the first framework that supports all main kinds of DNNs, from CNNs to RNNs, transformer, language models, and so on; the fastest DNN pruning and acceleration framework, up to 180X faster compared with current DNN pruning on other frameworks such as TensorFlow-Lite; making many representative AI applications able to run in real-time on off-the-shelf mobile devices that have been previously regarded possible only with special hardware support; making off-the-shelf mobile devices outperform a number of representative ASIC and FPGA solutions in terms of energy efficiency and/or performance.
References in corpus (12)
- Distilling the Knowledge in a Neural Network
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Do Deep Nets Really Need to be Deep?
- Pruning Convolutional Neural Networks for Resource Efficient Inference
- Binarized Neural Networks
- Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
- Fixed Point Quantization of Deep Convolutional Networks
- Wide Activation for Efficient and Accurate Image Super-Resolution
- Exploring the Regularity of Sparse Structure in Convolutional Neural Networks
- N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning
- Multi-style Generative Network for Real-time Transfer
- NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigm