Q-Rater: Non-Convex Optimization for Post-Training Uniform Quantization

Abstract

Various post-training uniform quantization methods have usually been studied based on convex optimization. As a result, most previous ones rely on the quantization error minimization and/or quadratic approximations. Such approaches are computationally efficient and reasonable when a large number of quantization bits are employed. When the number of quantization bits is relatively low, however, non-convex optimization is unavoidable to improve model accuracy. In this paper, we propose a new post-training uniform quantization technique considering non-convexity. We empirically show that hyper-parameters for clipping and rounding of weights and activations can be explored by monitoring task loss. Then, an optimally searched set of hyper-parameters is frozen to proceed to the next layer such that an incremental non-convex optimization is enabled for post-training quantization. Throughout extensive experimental results using various models, our proposed technique presents higher model accuracy, especially for a low-bit quantization.

References in corpus (11)

• Distilling the Knowledge in a Neural Network
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Trained Ternary Quantization
• The Loss Surfaces of Multilayer Networks
• Fixed Point Quantization of Deep Convolutional Networks
• Qualitatively characterizing neural network optimization problems
• Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
• Resiliency of Deep Neural Networks under Quantization
• And the Bit Goes Down: Revisiting the Quantization of Neural Networks
• Fighting Quantization Bias With Bias
• FleXOR: Trainable Fractional Quantization