FINN-R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks
arXiv:1809.04570
Abstract
Convolutional Neural Networks have rapidly become the most successful machine learning algorithm, enabling ubiquitous machine vision and intelligent decisions on even embedded computing-systems. While the underlying arithmetic is structurally simple, compute and memory requirements are challenging. One of the promising opportunities is leveraging reduced-precision representations for inputs, activations and model parameters. The resulting scalability in performance, power efficiency and storage footprint provides interesting design compromises in exchange for a small reduction in accuracy. FPGAs are ideal for exploiting low-precision inference engines leveraging custom precisions to achieve the required numerical accuracy for a given application. In this article, we describe the second generation of the FINN framework, an end-to-end tool which enables design space exploration and automates the creation of fully customized inference engines on FPGAs. Given a neural network description, the tool optimizes for given platforms, design targets and a specific precision. We introduce formalizations of resource cost functions and performance predictions, and elaborate on the optimization algorithms. Finally, we evaluate a selection of reduced precision neural networks ranging from CIFAR-10 classifiers to YOLO-based object detection on a range of platforms including PYNQ and AWS\,F1, demonstrating new unprecedented measured throughput at 50TOp/s on AWS-F1 and 5TOp/s on embedded devices.
arXiv commentto be published in ACM TRETS Special Edition on Deep Learning
References in corpus (14)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- Wide Residual Networks
- DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
- Learning both Weights and Connections for Efficient Neural Networks
- Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
- XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
- WRPN: Wide Reduced-Precision Networks
- Bitwise Neural Networks
- Resiliency of Deep Neural Networks under Quantization
- Deep Learning with Low Precision by Half-wave Gaussian Quantization
- Streamlined Deployment for Quantized Neural Networks