Rethinking Machine Learning Development and Deployment for Edge Devices
arXiv:1806.07846
Abstract
Machine learning (ML), especially deep learning is made possible by the availability of big data, enormous compute power and, often overlooked, development tools or frameworks. As the algorithms become mature and efficient, more and more ML inference is moving out of datacenters/cloud and deployed on edge devices. This model deployment process can be challenging as the deployment environment and requirements can be substantially different from those during model development. In this paper, we propose a new ML development and deployment approach that is specially designed and optimized for inference-only deployment on edge devices. We build a prototype and demonstrate that this approach can address all the deployment challenges and result in more efficient and high-quality solutions.
References in corpus (11)
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- cuDNN: Efficient Primitives for Deep Learning
- Hello Edge: Keyword Spotting on Microcontrollers
- Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations
- Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks
- Not All Ops Are Created Equal!
- Quantizing Convolutional Neural Networks for Low-Power High-Throughput Inference Engines