Performance-Efficiency Trade-off of Low-Precision Numerical Formats in Deep Neural Networks

Abstract

Deep neural networks (DNNs) have been demonstrated as effective prognostic models across various domains, e.g. natural language processing, computer vision, and genomics. However, modern-day DNNs demand high compute and memory storage for executing any reasonably complex task. To optimize the inference time and alleviate the power consumption of these networks, DNN accelerators with low-precision representations of data and DNN parameters are being actively studied. An interesting research question is in how low-precision networks can be ported to edge-devices with similar performance as high-precision networks. In this work, we employ the fixed-point, floating point, and posit numerical formats at $\le$8-bit precision within a DNN accelerator, Deep Positron, with exact multiply-and-accumulate (EMAC) units for inference. A unified analysis quantifies the trade-offs between overall network efficiency and performance across five classification tasks. Our results indicate that posits are a natural fit for DNN inference, outperforming at $\le$8-bit precision, and can be realized with competitive resource requirements relative to those of floating point.

arXiv comment9 pages, Proceedings of the ACM Conference for Next Generation Arithmetic (CoNGA) 2019

References in corpus (3)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Rethinking floating point for deep learning
• Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks

Cited by in corpus (7)

• Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques
• PositNN: Training Deep Neural Networks with Mixed Low-Precision Posit
• Accuracy-Efficiency Trade-Offs and Accountability in Distributed ML Systems
• EvGNN: An Event-driven Graph Neural Network Accelerator for Edge Vision
• Template-Based Posit Multiplication for Training and Inferring in Neural Networks
• Deep Learning Training on the Edge with Low-Precision Posits
• An Automotive Case Study on the Limits of Approximation for Object Detection