Confounding Tradeoffs for Neural Network Quantization
arXiv:2102.06366
Abstract
Many neural network quantization techniques have been developed to decrease the computational and memory footprint of deep learning. However, these methods are evaluated subject to confounding tradeoffs that may affect inference acceleration or resource complexity in exchange for higher accuracy. In this work, we articulate a variety of tradeoffs whose impact is often overlooked and empirically analyze their impact on uniform and mixed-precision post-training quantization, finding that these confounding tradeoffs may have a larger impact on quantized network accuracy than the actual quantization methods themselves. Because these tradeoffs constrain the attainable hardware acceleration for different use-cases, we encourage researchers to explicitly report these design choices through the structure of "quantization cards." We expect quantization cards to help researchers compare methods more effectively and engineers determine the applicability of quantization techniques for their hardware.
References in corpus (10)
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Quantizing deep convolutional networks for efficient inference: A whitepaper
- PACT: Parameterized Clipping Activation for Quantized Neural Networks
- Fixed Point Quantization of Deep Convolutional Networks
- Convolutional Neural Networks using Logarithmic Data Representation
- Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
- The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards
- Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming
- Mixed Precision DNNs: All you need is a good parametrization
- Precision Highway for Ultra Low-Precision Quantization